Splunk Search

Has anyone done hardware benchmarking with Splunk and these m2 interface disks?

daniel333
Builder

All,

Has anyone done any hardware benchmarking with splunk and these m2 interface disks?
http://www.tomshardware.com/reviews/kingston-hyperx-predator-480gb-m2-pcie-ssd,4113.html#p4

Seems like they would be pretty performant in a cluster.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Obviously all the following is my opinion plus what I've heard others say. I have all SSD based indexers and they're fast, but not as fast as one might think.

In general, SSDs of pretty much any non-terrible sort (and maybe even the "terrible" ones) are all pretty capable.

If you take a reasonably close look at a Splunk indexer in action, you will find that there's a balancing act going on between CPU and disk. If your disk is slow it generally will be your bottleneck because the CPU will be more often than not waiting on your disks to fetch or save data. If the speed with which you can access your disks gets improved, the faster disks will reduce the bottleneck they were creating.

A similar thing happens with CPU. Suppose you are running SSDs underneath a slow two core system. In that case, your disks are rarely if ever the holdup, instead you are usually waiting on your CPU. If you increase the speed of your CPU, you will increase the load on your disks up until they both CPU and disk are saturated.

So, sticking SSDs under an indexer can certainly improve performance since fast CPU seems to be cheaper to stuff into a box than fast disk. But it can only do so up to the point where your CPU is now the limiting factor.

This really depends on the workload of course, but in general I'd say the improvement in an "overall" type of performance metric would put an SSD-based indexer at capable of ingesting about twice as much data, assuming you aren't running a Splunk workload that is already hammering your CPU.

Now, to the main point. Those particular SSDs - I didn't see if they were particularly fast or not, but in my opinion that doesn't matter much. Pretty much ANY solid, decent SSD should be fast enough to move a lot of your bottleneck to CPU. Switching from spinning platters to SSD makes a nice improvement in performance. Switching from one SSD to another probably wouldn't be noticeable. It might, but probably not. In any case, it's a far lower performance improvement than the one of going from spinning rust to SSD.

But again, it all depends on your own workload. Some workloads will be fantastically faster with SSD, others will hardly be any different - it all depends on what the system was doing in the first place.

We buy our SSDs based on support, warranty and reliability and match their capabilities to the workload we've found we have. The last percent of performance isn't worth chasing after because we'd rather have stuff we can trust day in and day out with the load we're placing on it and have a reliable support plan in place for when they fail.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...