Getting Data In

Hardware Requirements For Splunk Deployment Master & Splunk Heavy Forwarder Servers

anandhalagaras1
Communicator

Hi Team,

We are planning to host the Deployment Master server and two Splunk Heavy Forwarder servers in our on-prem Nutanix environment. Could you please provide the recommended hardware requirements for hosting these servers? Based on your input, we will plan and provision the necessary hardware.

The primary role of the Deployment Master server will be to create custom apps and collect data from client machines using Splunk Universal Forwarder.

For the Heavy Forwarders, we will be installing multiple add-ons to configure and fetch data from sources such as Azure Storage (Table, Blob), O365 applications, Splunk DB Connect, Qualys, AWS, and client machine data parsing.

We are looking for the minimum, moderate, and maximum hardware requirements as recommended by Splunk Support to host the Splunk DM and HF servers in the Nutanix environment. If there are any support articles or documentation available, that would be greatly appreciated.

Thank you!

Labels (4)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

There is no single good answer to such question.

A Deployment Server (not Deployment Master), depending on your environment size and configuration parameters, can run perfectly well on a relatively small server (like 4CPU and 8GB; if you disable GUI, probably even smaller) but can need to be load-balanced over several quite big machines if you have many clients and many often changing apps.

As for HF, good thing is that you don't have to have just one HF in your environment (technically, you can have multiple separate DS instances for separate segments of your deployment but it makes app management more troublesome).. So you can start with a moderately sized HF (like a reference all-in-one server) and either scale out by adding cores/memory if you start lacking resources or add more instances of HF and migrate some inputs there.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @anandhalagaras1 ,

there isn't any formal requirement from Splunk about Deployment Server and Heavy Forwarders, the only requirements are for a normal stand-alone Splunk Server: 12 CPUs and 12 GB RAM.

From my experience, I could add that, for DS, it depends on the number of client, if they aren't so many (some hundreds), you could also have less CPUs and RAM (8+8), in addition, from few time, you can also use more than one DS.

It's different for HFs: if they have to do an hard job for parsing logs (regexes), it's better to give them more resources (expecially CPUs); in one heavy project, where our 4 HF had to receive and parse hundreds of GB every day, I used 24 CPUs and 64 GB RAM for each one.

My hint is to start with the normal reference hardware (12+12), analyze machine loads and queues and eventually add more resources (we're usually speaking of virtual servers).

In addition, if you have to receive syslogs, don't use Splunk for them, but use an rsyslog (or syslog-ng) server and then Splunk can read the written files.

Ciao.

Giuseppe

0 Karma

anandhalagaras1
Communicator

@gcusello , Thank you for your swift response.

For the Deployment Master server, we have around 1,000+ client machines in our environment. So it would be helpful if you could help me with the recommended hardware specifications for this setup?

As for the Heavy Forwarders, we will be ingesting over 40 GB of approximate data daily from both the HF servers. The primary data sources include Microsoft Azure Storage Table and Blob using the Splunk Add-On for Microsoft Cloud Services, the Qualys Technology Add-On, Splunk DB Connect, and data parsing for approximately 120+ client machines per Heavy Forwarder. What would be the recommended hardware specifications for these servers?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @anandhalagaras1 ,

as I said, start with the default configuration (12CPUs and 12 GB RAM) and analyze the machine load using the monitoring console and the queues.

If you haven't too high queues and too high load maintain the default configuration, otherwise, add more resources, It isn't possible to give a general configuration.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

Industry Solutions for Supply Chain and OT, Amazon Use Cases, Plus More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...