topic Re: Splunk enterprise sizing with ES in Splunk Search

Splunk enterprise sizing with ES

hariskhan — Wed, 09 Jan 2019 05:11:04 GMT

Hello everybody,
I am sizing hardware for splunk enterprise and enterprise security solution.
We are designing that for 80GB/day data for Splunk enterprise and enterprise security and did following hardware sizing for 6 months data retention. We kept in view the HA factor as well.

Search Heads x3

Memory 16GB

Onbox storage: 1TB X 2 Raid 1
Processor 8Core X 2 @ 2.1 GHz
RAID controller yes
Power AC

PC dual 2 port 16GB

NIC 1G X4 etnernet

Indexersx3
Memory 16GB
Onbox storage 1TB X 2 Raid 1
Processor 8Core X 2 @ 2.1 GHz
RAID controller yes
Power AC
FC card dual 2 port 16GB

NIC 1G X4 etnernet

Master Server x1
Memory 16GB

Onbox storage 500GB X 2 Raid 1
Processor 8Core X 2 @ 2.1GHz
RAID controller yes
Power AC

FC card dual 2 port 16GB

NIC 1G X4 etnernet

Heavy Forwarders x 2

Memory 16GB
Onbox storage 500GB X 2 Raid 1
Processor 8Core X 1 @ 2.1GHz
Raid Controller yes
Power AC dual

NIC 1G X4 etnernet

SAN

30TB SAN storage with 2 SAN switches. RAID 10 OR 1

Plan is to make SH cluster and indexer cluster.Master server is also a deployment server.
Can someone advice whether above sizing will be adequate for 75GB/day data when used with splunk entperise and enterprise security, In not please advice on any incremental changes?.
Can above solution be able to run 4 concurrent searches on dashboard without service deterioration.

Re: Splunk enterprise sizing with ES

lakshman239 — Wed, 09 Jan 2019 13:05:36 GMT

What's your designed Search factor (SF) and Replication factor( RF). Do you have another instance/server acting as 'deployer'? (to manage config for SHC?)

Have you thought of which correlation searches would you be turning on in the Enterprise Security (ES)? (as this will use concurrent searches in addition to your users, scheduled jobs etc..)

ES uses datamodels and based on the amount of data which you have in the datamodel acceleration, it will consume additional storage in the indexing tier. that needs to be factored in based on the datamodels planned to be used/correlation searches enabled.

You can also check this to get a some idea/approach - https://splunk-sizing.appspot.com/

Re: Splunk enterprise sizing with ES

hariskhan — Wed, 09 Jan 2019 16:44:28 GMT

What's your designed Search factor (SF) and Replication factor( RF). Do you have another instance/server acting as 'deployer'? (to manage config for SHC?)

Replication factor =3 since i have 3 SH and 3 INDXers, Serach head cluster is also 3.

I have not added deployer , thanks for info i will add that.I will also be adding deployment server.Will Appreciate if you can mention the recommend specs for both servers.

Have you thought of which correlation searches would you be turning on in the Enterprise Security (ES)? (as this will use concurrent searches in addition to your users, scheduled jobs etc..)

I have not decided that yet . Need details on that if you can point me to some doc that relates that to hardware sizing.

How to account for storage requirement needed for ES data models.

I have used same link as mentioned by you , for sizing and it says i will be needing 30TB storage.

Do i need to add additional cores or RAM to indexers or Search heads for Enterprise security application?.

Re: Splunk enterprise sizing with ES

hariskhan — Fri, 11 Jan 2019 06:38:42 GMT

any update on this ?.

Re: Splunk enterprise sizing with ES

woodcock — Sat, 12 Jan 2019 01:03:31 GMT

The general rule of thumb for non-clustered Indexers for ES is NO MORE than 100GB/indexer. I would add 10% indexers if you are going to use clustering. So you are fine.

Re: Splunk enterprise sizing with ES

hariskhan — Sat, 12 Jan 2019 03:49:55 GMT

Thanks for the help.

Re: Splunk enterprise sizing with ES

SyaloomKris — Wed, 17 May 2023 06:58:16 GMT

Hi splunk team,

Need confirmation, how many sizing that my company need that we will integrate to splunk siem ?

there are 104 source log, with 30 days log retention