Deployment Architecture

Indexes.conf Stanza Storage for Indexer Cluster Settings Calculation

keithevanscdcr
Explorer

Hello all. Thanks in advance for your assistance.

I have a 6 node index cluster with a search factor of 6 and replication factor of 3. My ingest is 130GB per/day for proxy data
My Hardware: 7TB hot/warm SSD and 112TB cold 10k spindle per/indexer
I have a requirement for 30 days hot/warm and 1,065 days cold = 1,095 days (3 years) total

Calculations I have found say:
(Daily Avg. Ingest x Compression Rate x Num Days Retention) / # of Indexers

So:
Hot/Warm: 130GB * .6 compression * 15 days) / 6 = 390 GB p/indexer
Cold: 130GB * .6 compression * 1,065 days) / 6 = 13,845 GB p/indexer
Total: 130GB * .6 compression * 1,095 days) / 6 = 14,235 GB p/indexer

Given that, I think my indexes.conf would have:
[idx_proxy]
homePath = volume:primary/idx_proxy/db
coldPath = volume:secondary/idx_proxy/colddb
thawedPath = $SPLUNK_DB/idx_proxy/thaweddb
maxTotalDataSizeMB = 14576640
maxDataSize = auto_high_volume
homePath.maxDataSizeMB = 399360
frozenTimePeriodInSecs = 94608000

The question: With replication factor, I'm thinking this answer is not complete. Can anyone help? Updated formula/calc?

This is current storage:
[splunk@splunkidx1 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
947G /splunkData/hot/idx_proxy/

[splunk@splunkidx2 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
826G /splunkData/hot/idx_proxy/

[splunk@splunkidx3 ~]$
7.9T /splunkData/cold/idx_proxy/
955G /splunkData/hot/idx_proxy/

[splunk@splunkidx4 ~]$ du -sh /splunkData/*/idx_proxy/
7.8T /splunkData/cold/idx_proxy/
952G /splunkData/hot/idx_proxy/

[splunk@splunkidx5 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
936G /splunkData/hot/idx_proxy/

[splunk@splunkidx6 ~]$ du -sh /splunkData/*/idx_proxy/
7.8T /splunkData/cold/idx_proxy/
911G /splunkData/hot/idx_proxy/

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Announcing the Expansion of the Splunk Academic Alliance Program

The Splunk Community is more than just an online forum — it’s a network of passionate users, administrators, ...

Learn Splunk Insider Insights, Do More With Gen AI, & Find 20+ New Use Cases You Can ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Buttercup Games: Further Dashboarding Techniques (Part 7)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...