Deployment Architecture

What are the hardware requirements for a cluster master?

BobM
Builder

I have read Managing Indexers and Clusters trying to find what the requirements are for a cluster master. It says 'The hardware storage needs of the master node are obviously lower than those specified in the "Reference hardware" topic, since the master does not index external data.'

But surely a master node doesn't need 12 cores to manage 4 indexers and a search head. So what factors do we have to take in to account.

dwaddle
SplunkTrust
SplunkTrust

Let's update this for 2019! It seems to come up a lot in google results, and since I originally wrote this in 2015 "things have changed". Splunk environments are larger and more complex, and honestly I know more about scaling clusters than I did then. A very small cluster (3-4 indexers, 500GB a day or so) would likely be fine with the below specs.

In 2019, I would make sure that my cluster master is similar in specs to my indexers - except for the disk space requirement. The CM does not need a lot of disk, but it can use lots of CPU and lots of RAM. Every bucket in the cluster has state that must be kept in the CM's RAM, and the status of those buckets must be updated as things on the cluster happen. There's a saying, "More Buckets More Problems" so be aware that CM scaling and tuning is partly a function of the number of buckets in your cluster.

FIrst of all you should follow what the Splunk docs say as far as hardware requirements! Beyond that, a good reference is Da Xu's and Chloe Yeung's .conf talk "Indexer Clustering Internals, Scaling and Performance Testing". See the slides and video from .conf 2018.

Old answer from 2015 follows, please don't try to use this unless you're reading this from a time machine in 2015:

The hardware requirements for a cluster manager really are small. Search concurrency does play a part as does the number of indexers. The primary role of the cluster master is to coordinate which indexer houses the "primary" copies of buckets and which indexer(s) hold the "backup" copies. If your cluster master is dedicated exclusively to this task (and it should be) then any small-to-medium size cluster should be able to get by with 2 cores and 4GB of RAM. Even if the rest of your Splunk installation is on "bare metal", a cluster master is a good candidate for a small virtual machine.

brod_geico
Path Finder

In general every thing is come to your user count,Max concurrence searches running, how many reports, how much data load all will come into picture.
As general recommendation is below one Master distributed the data.
4xquad-core CPUsat2.0 GHz/12Gb ~ 16GB RAM/ 2x10kRPMlocal SAS drives in RAID 1
To test your load please make sure your run Bonee++ utility on linux servers.

0 Karma

adrianathome
Communicator

I have been looking for an answer to this myself. Currently, I am using the following: 2 vcpus, 2GB RAM, 25 GB Storage. My cluster is <1 day old and no issues. I did notice that the cluster master has access to the events that get sent to search peers. Essentially, the master appears to function as a search head. If I find out I'll let you know. BTW my cluster is not in production. I am just sharing my observations so far.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

&#x1f342; Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...