You appear to have at least two problems:
1) Your data is not evenly distributed across your indexers. Even distribution would have kept the 29 drives from filling up quickly and would improve search performance, but is not your main problem.
2) Your indexes are mis-configured. Volumes should be sized so they don't, combined, exceed the available storage. Don't forget to allow for file system overhead, data model accelerations, and replicated buckets. We'd have to know more about your index configuration to offer specific advise.
Also. you may have too many replicated buckets. Consider lowering your replication factor.
Make sure $SPLUNK_DB is not sharing storage with $SPLUNK_HOME, the operating system, or another application.
We are set to 2 searchable and 3 replicated right now. I believe we are sized too small. We have 2.7 gb of space on all the indexers and 2.6 is used. I think it was configure to have 30 days of searchable data in HOT and I think that is too much data. Do you think increasing the storage of the indexers is the best option or decrease the days os HOT searchable?
Adding more storage is the best idea, but you may find yourself in the same situation later if you don't get your configuration right. Once you have the settings tuned buckets should roll before the storage fills.