Hardware Requirements For Splunk Deployment Master & Splunk Heavy Forwarder Servers

[anandhalagaras1]

Hi Team,

We are planning to host the Deployment Master server and two Splunk Heavy Forwarder servers in our on-prem Nutanix environment. Could you please provide the recommended hardware requirements for hosting these servers? Based on your input, we will plan and provision the necessary hardware.

The primary role of the Deployment Master server will be to create custom apps and collect data from client machines using Splunk Universal Forwarder.

For the Heavy Forwarders, we will be installing multiple add-ons to configure and fetch data from sources such as Azure Storage (Table, Blob), O365 applications, Splunk DB Connect, Qualys, AWS, and client machine data parsing.

We are looking for the minimum, moderate, and maximum hardware requirements as recommended by Splunk Support to host the Splunk DM and HF servers in the Nutanix environment. If there are any support articles or documentation available, that would be greatly appreciated.

Thank you!

[PickleRick]

There is no single good answer to such question.

A Deployment Server (not Deployment Master), depending on your environment size and configuration parameters, can run perfectly well on a relatively small server (like 4CPU and 8GB; if you disable GUI, probably even smaller) but can need to be load-balanced over several quite big machines if you have many clients and many often changing apps.

As for HF, good thing is that you don't have to have just one HF in your environment (technically, you can have multiple separate DS instances for separate segments of your deployment but it makes app management more troublesome).. So you can start with a moderately sized HF (like a reference all-in-one server) and either scale out by adding cores/memory if you start lacking resources or add more instances of HF and migrate some inputs there.

[gcusello]

Hi @anandhalagaras1 ,

there isn't any formal requirement from Splunk about Deployment Server and Heavy Forwarders, the only requirements are for a normal stand-alone Splunk Server: 12 CPUs and 12 GB RAM.

From my experience, I could add that, for DS, it depends on the number of client, if they aren't so many (some hundreds), you could also have less CPUs and RAM (8+8), in addition, from few time, you can also use more than one DS.

It's different for HFs: if they have to do an hard job for parsing logs (regexes), it's better to give them more resources (expecially CPUs); in one heavy project, where our 4 HF had to receive and parse hundreds of GB every day, I used 24 CPUs and 64 GB RAM for each one.

My hint is to start with the normal reference hardware (12+12), analyze machine loads and queues and eventually add more resources (we're usually speaking of virtual servers).

In addition, if you have to receive syslogs, don't use Splunk for them, but use an rsyslog (or syslog-ng) server and then Splunk can read the written files.

Ciao.

Giuseppe

[anandhalagaras1]

@gcusello , Thank you for your swift response.

For the Deployment Master server, we have around 1,000+ client machines in our environment. So it would be helpful if you could help me with the recommended hardware specifications for this setup?

As for the Heavy Forwarders, we will be ingesting over 40 GB of approximate data daily from both the HF servers. The primary data sources include Microsoft Azure Storage Table and Blob using the Splunk Add-On for Microsoft Cloud Services, the Qualys Technology Add-On, Splunk DB Connect, and data parsing for approximately 120+ client machines per Heavy Forwarder. What would be the recommended hardware specifications for these servers?

, 

[gcusello]

Hi @anandhalagaras1 ,

as I said, start with the default configuration (12CPUs and 12 GB RAM) and analyze the machine load using the monitoring console and the queues.

If you haven't too high queues and too high load maintain the default configuration, otherwise, add more resources, It isn't possible to give a general configuration.

Ciao.

Giuseppe