Deployment Architecture

Can I mix different types of disks for search head and indexers?

wganesh28
Explorer

Hello,

Can I mix different type of disks, for example SSDs and HDDs, while installing search heads or indexers, in on-premise environment

Best regards, 

 

Labels (1)
0 Karma

wganesh28
Explorer

Thanks Rich Galloway. I am assuming the SSDs and HDDs can be put in the same server. Is there any link where this is is mentioned?

Also I am looking for details of hot/warm/cold buckets, is this a good starting point: https://docs.splunk.com/Documentation/Splunk/9.0.3/Indexer/HowSplunkstoresindexes

There is a section - "What the index directories look like"

I don't think we need any such arrangement for search head, is my assumption correct? 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, you can mix SSDs and HDDs on the same server.  Splunk doesn't care as long as the IOPS meet the minimum requirements.  The recommendation is the fastest disk(s), usually SSDs, be used for writing (hot buckets) and the most-frequently used data.  Everything else can go on slower disks, usually HDDs.

The details about buckets are good to know, but aren't much of a factor in the hardware, aside from the above.

Do keep your Splunk directories on separate mount points from the OS.

The SH does not have the same I/O demands as the indexers.

---
If this reply helps you, Karma would be appreciated.

wganesh28
Explorer

"Do keep your Splunk directories on separate mount points from the OS." - Yes that is the idea. We haven't implemented this yet though. 

Regarding search head - my idea is to keep read intensive operations (such as querying particular logs etc.) on SSDs, and allocate HDDs for offline report generation using Splunk pipeline/job features. I want to use SSDs for read intensive operations, and I want to use HDDs for read/write operations to reduce component failures. 

Now for indexers, Splunk has clearly outlined the policy for hot/warm/cold buckets, however for search heads, I see hints. Do you see any reference implementation for different types of disks in same server for indexers and search heads? 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Remember that Splunk indexes are write-once-read-many files so only hot buckets have write operations (other than saving of knowledge bundles from the SH).  Most searches are read-intensive so put the data accessed most often (usually < 30 days old) on SSDs.

A lot of work has gone into the effects of storage speed on indexer performance, but not so much on SH performance.  That's probably because indexers are much more I/O intensive and SHs.  I'd focus on the indexers and then put the leftover drives on the SHs.

---
If this reply helps you, Karma would be appreciated.

richgalloway
SplunkTrust
SplunkTrust

Yes, you can.  Splunk recommends you use SSD for hot/warm buckets and HDD for cold buckets.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...