With our experience with storage we started with 10-16x 15K RPM SAS drives in our indexers which depending on your indexing volume worked pretty well, however we moved our clustered indexers hot DBs over to storage on our SSD SAN arrays where we saw a pretty significant increase in performance (2x-5x). It seems to come down to how Splunk searches across the buckets, it generates random IO against the disks while looking through the indexes, with SSD storage you start to see big improvements with sub millisecond latency and high random IO that the SSDs can deliver since data is returned much quicker, even if you have a high write rate to the indexes and a lot of searches going on in the system.
One other option for you would be to put a couple SSDs into the system to handle your hot buckets then some larger 10K RPM drives in the system to handle your cold buckets. Just make sure you have enough SSD based storage to handle the majority of your searches and that you don't dip into the cold buckets as often on the slower disks (This is how we are configured).
... View more