Hello,
Can I mix different type of disks, for example SSDs and HDDs, while installing search heads or indexers, in on-premise environment?
Best regards,
Thanks Rich Galloway. I am assuming the SSDs and HDDs can be put in the same server. Is there any link where this is is mentioned?
Also I am looking for details of hot/warm/cold buckets, is this a good starting point: https://docs.splunk.com/Documentation/Splunk/9.0.3/Indexer/HowSplunkstoresindexes
There is a section - "What the index directories look like"
I don't think we need any such arrangement for search head, is my assumption correct?
Yes, you can mix SSDs and HDDs on the same server. Splunk doesn't care as long as the IOPS meet the minimum requirements. The recommendation is the fastest disk(s), usually SSDs, be used for writing (hot buckets) and the most-frequently used data. Everything else can go on slower disks, usually HDDs.
The details about buckets are good to know, but aren't much of a factor in the hardware, aside from the above.
Do keep your Splunk directories on separate mount points from the OS.
The SH does not have the same I/O demands as the indexers.
"Do keep your Splunk directories on separate mount points from the OS." - Yes that is the idea. We haven't implemented this yet though.
Regarding search head - my idea is to keep read intensive operations (such as querying particular logs etc.) on SSDs, and allocate HDDs for offline report generation using Splunk pipeline/job features. I want to use SSDs for read intensive operations, and I want to use HDDs for read/write operations to reduce component failures.
Now for indexers, Splunk has clearly outlined the policy for hot/warm/cold buckets, however for search heads, I see hints. Do you see any reference implementation for different types of disks in same server for indexers and search heads?
Remember that Splunk indexes are write-once-read-many files so only hot buckets have write operations (other than saving of knowledge bundles from the SH). Most searches are read-intensive so put the data accessed most often (usually < 30 days old) on SSDs.
A lot of work has gone into the effects of storage speed on indexer performance, but not so much on SH performance. That's probably because indexers are much more I/O intensive and SHs. I'd focus on the indexers and then put the leftover drives on the SHs.
Yes, you can. Splunk recommends you use SSD for hot/warm buckets and HDD for cold buckets.