Splunk Search

Splunk enterprise sizing with ES

hariskhan
Explorer

Hello everybody,
I am sizing hardware for splunk enterprise and enterprise security solution.
We are designing that for 80GB/day data for Splunk enterprise and enterprise security and did following hardware sizing for 6 months data retention. We kept in view the HA factor as well.

Search Heads x3

Memory 16GB

Onbox storage: 1TB X 2 Raid 1
Processor 8Core X 2 @ 2.1 GHz
RAID controller yes
Power AC

PC dual 2 port 16GB

NIC 1G X4 etnernet

Indexersx3
Memory 16GB
Onbox storage 1TB X 2 Raid 1
Processor 8Core X 2 @ 2.1 GHz
RAID controller yes
Power AC
FC card dual 2 port 16GB

NIC 1G X4 etnernet

Master Server x1
Memory 16GB

Onbox storage 500GB X 2 Raid 1
Processor 8Core X 2 @ 2.1GHz
RAID controller yes
Power AC

FC card dual 2 port 16GB

NIC 1G X4 etnernet

Heavy Forwarders x 2

Memory 16GB
Onbox storage 500GB X 2 Raid 1
Processor 8Core X 1 @ 2.1GHz
Raid Controller yes
Power AC dual

NIC 1G X4 etnernet

SAN

30TB SAN storage with 2 SAN switches. RAID 10 OR 1

Plan is to make SH cluster and indexer cluster.Master server is also a deployment server.
Can someone advice whether above sizing will be adequate for 75GB/day data when used with splunk entperise and enterprise security, In not please advice on any incremental changes?.
Can above solution be able to run 4 concurrent searches on dashboard without service deterioration.

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

The general rule of thumb for non-clustered Indexers for ES is NO MORE than 100GB/indexer. I would add 10% indexers if you are going to use clustering. So you are fine.

View solution in original post

0 Karma

woodcock
Esteemed Legend

The general rule of thumb for non-clustered Indexers for ES is NO MORE than 100GB/indexer. I would add 10% indexers if you are going to use clustering. So you are fine.

0 Karma

hariskhan
Explorer

Thanks for the help.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

What's your designed Search factor (SF) and Replication factor( RF). Do you have another instance/server acting as 'deployer'? (to manage config for SHC?)

Have you thought of which correlation searches would you be turning on in the Enterprise Security (ES)? (as this will use concurrent searches in addition to your users, scheduled jobs etc..)

ES uses datamodels and based on the amount of data which you have in the datamodel acceleration, it will consume additional storage in the indexing tier. that needs to be factored in based on the datamodels planned to be used/correlation searches enabled.

You can also check this to get a some idea/approach - https://splunk-sizing.appspot.com/

0 Karma

hariskhan
Explorer

What's your designed Search factor (SF) and Replication factor( RF). Do you have another instance/server acting as 'deployer'? (to manage config for SHC?)

Replication factor =3 since i have 3 SH and 3 INDXers, Serach head cluster is also 3.

I have not added deployer , thanks for info i will add that.I will also be adding deployment server.Will Appreciate if you can mention the recommend specs for both servers.

Have you thought of which correlation searches would you be turning on in the Enterprise Security (ES)? (as this will use concurrent searches in addition to your users, scheduled jobs etc..)

I have not decided that yet . Need details on that if you can point me to some doc that relates that to hardware sizing.

How to account for storage requirement needed for ES data models.

I have used same link as mentioned by you , for sizing and it says i will be needing 30TB storage.

Do i need to add additional cores or RAM to indexers or Search heads for Enterprise security application?.

0 Karma

SyaloomKris
Observer

Hi splunk team,

 

Need confirmation, how many sizing that my company need that we will integrate to splunk siem ?

there are 104 source log, with 30 days log retention

 

 

0 Karma

hariskhan
Explorer

any update on this ?.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...