Hello all. Thanks in advance for your assistance.
I have a 6 node index cluster with a search factor of 6 and replication factor of 3. My ingest is 130GB per/day for proxy data
My Hardware: 7TB hot/warm SSD and 112TB cold 10k spindle per/indexer
I have a requirement for 30 days hot/warm and 1,065 days cold = 1,095 days (3 years) total
Calculations I have found say:
(Daily Avg. Ingest x Compression Rate x Num Days Retention) / # of Indexers
So:
Hot/Warm: 130GB * .6 compression * 15 days) / 6 = 390 GB p/indexer
Cold: 130GB * .6 compression * 1,065 days) / 6 = 13,845 GB p/indexer
Total: 130GB * .6 compression * 1,095 days) / 6 = 14,235 GB p/indexer
Given that, I think my indexes.conf would have:
[idx_proxy]
homePath = volume:primary/idx_proxy/db
coldPath = volume:secondary/idx_proxy/colddb
thawedPath = $SPLUNK_DB/idx_proxy/thaweddb
maxTotalDataSizeMB = 14576640
maxDataSize = auto_high_volume
homePath.maxDataSizeMB = 399360
frozenTimePeriodInSecs = 94608000
The question: With replication factor, I'm thinking this answer is not complete. Can anyone help? Updated formula/calc?
This is current storage:
[splunk@splunkidx1 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
947G /splunkData/hot/idx_proxy/
[splunk@splunkidx2 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
826G /splunkData/hot/idx_proxy/
[splunk@splunkidx3 ~]$
7.9T /splunkData/cold/idx_proxy/
955G /splunkData/hot/idx_proxy/
[splunk@splunkidx4 ~]$ du -sh /splunkData/*/idx_proxy/
7.8T /splunkData/cold/idx_proxy/
952G /splunkData/hot/idx_proxy/
[splunk@splunkidx5 ~]$ du -sh /splunkData/*/idx_proxy/
8.0T /splunkData/cold/idx_proxy/
936G /splunkData/hot/idx_proxy/
[splunk@splunkidx6 ~]$ du -sh /splunkData/*/idx_proxy/
7.8T /splunkData/cold/idx_proxy/
911G /splunkData/hot/idx_proxy/
