Performance in Virtual versus Hardware Indexers for large and growing Enterprise Splunk instantiations

[swagner1965]

We have an Enterprise Splunk instantiation that has clustered virtual indexers.  We have been advised that we need real hardware for our indexers to scale up to the size we anticipate.  What areas of performance are affected by having virtualized indexers versus hardware?  

[Stefanie · Accepted solution]

The main issue that virtual Splunk servers have is the fact that the resources Splunk needs is not "reserved".

Virtual Indexers perform best when their vCPU and vRAM is reserved, and the disk is provisioned using eager-zero. 

There was an old Splunk tech brief from 2017 that talked about deploying Splunk on Virtual Hardware. I will paste the summary below.

As is expected with most virtualized high I/O applications, you should expect as much as 10 percent less performance when running Splunk Enterprise within virtual environments. However, there are many additional benefits to consider. Virtualization offers better resource sharing and utilization, includes HA capabilities, makes provisioning and management an easier exercise, and may support a corporate virtualization mandate. For best performance, put full reservations on CPU and memory, provision Eager Zero Thick VMDKs, and turn off snapshotting for virtual machines running Splunk Enterprise. Disk quality is also critical to Splunk performance—make sure you are using the best disk available. And to keep up with increasing data volumes, scale your deployment by adding additional Splunk indexers.

[swagner1965]

Thanks!

That confirms what we have heard from conversations with other people and you referenced some documentation which will help us plead our case to the folks we plead to,.....

Cheers!