Deployment Architecture

Indexes.conf Stanza Storage for Indexer Cluster Settings Calculation


Hello all. Thanks in advance for your assistance.

I have a 6 node index cluster with a search factor of 6 and replication factor of 3. My ingest is 130GB per/day for proxy data
My Hardware: 7TB hot/warm SSD and 112TB cold 10k spindle per/indexer
I have a requirement for 30 days hot/warm and 1,065 days cold = 1,095 days (3 years) total

Calculations I have found say:
(Daily Avg. Ingest x Compression Rate x Num Days Retention) / # of Indexers

Hot/Warm: 130GB * .6 compression * 15 days) / 6 = 390 GB p/indexer
Cold: 130GB * .6 compression * 1,065 days) / 6 = 13,845 GB p/indexer
Total: 130GB * .6 compression * 1,095 days) / 6 = 14,235 GB p/indexer

Given that, I think my indexes.conf would have:
homePath = volume:primary/idx_proxy/db
coldPath = volume:secondary/idx_proxy/colddb
thawedPath = $SPLUNK_DB/idx_proxy/thaweddb
maxTotalDataSizeMB = 14576640
maxDataSize = auto_high_volume
homePath.maxDataSizeMB = 399360
frozenTimePeriodInSecs = 94608000

The question: With replication factor, I'm thinking this answer is not complete. Can anyone help? Updated formula/calc?

This is current storage:
[splunk@splunkidx1 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
947G /splunkData/hot/idx_proxy/

[splunk@splunkidx2 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
826G /splunkData/hot/idx_proxy/

[splunk@splunkidx3 ~]$
7.9T /splunkData/cold/idx_proxy/
955G /splunkData/hot/idx_proxy/

[splunk@splunkidx4 ~]$ du -sh /splunkData/*/idx_proxy/
7.8T /splunkData/cold/idx_proxy/
952G /splunkData/hot/idx_proxy/

[splunk@splunkidx5 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
936G /splunkData/hot/idx_proxy/

[splunk@splunkidx6 ~]$ du -sh /splunkData/*/idx_proxy/
7.8T /splunkData/cold/idx_proxy/
911G /splunkData/hot/idx_proxy/

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Splunk Lantern | Getting Started with Edge Processor, Machine Learning Toolkit ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 2 releases of new security content via the ...

Announcing the 1st Round Champion’s Tribute Winners of the Great Resilience Quest

We are happy to announce the 20 lucky questers who are selected to be the first round of Champion's Tribute ...