Hi,
We have cluster indexer setup with 5 indexers on separate ESX Servers each with 12TB HDD and 128GB RAM.
The cluster replication factor(RF) is 2 and Search factor(SF) is 1. We have one Job scheduler and search head and forwarder nodes.
Splunk version 6.5.3
Our daily log volume is close to 1TB/day. Out of 1TB log volume 85% of data goes to abc_transaction indexes (RAW data).
In our indexes.conf file we have set frozenTimePeriodInSecs = 3888000 (45 days) for abc_transaction RAW index.
Even though we have set frozenTimePeriodInSecs as 45 days, it is able to search only last 10 days of abc_transaction RAW data.
So to keep the abc_transaction data available for 45 days, we have gone through this Splunk sizing link and added maxVolumeDataSizeMB = 5976884 (5.7TB) attribute for abc_transaction index.
https://splunk-sizing.appspot.com/#ar=0&c=1&cr=30&ds=4096&hwr=60&i=5&rf=2&rl=10,01&sf=1&v=1000
indexes.conf
[abc_transaction]
repFactor=auto
coldPath = $SPLUNK_DB/abc_transaction/colddb
homePath = $SPLUNK_DB/abc_transaction/db
thawedPath = $SPLUNK_DB/abc_transaction/thaweddb
maxMemMB = 20
maxConcurrentOptimizes = 6
maxHotIdleSecs = 86400
maxHotBuckets = 10
maxDataSize = auto_high_volume
frozenTimePeriodInSecs = 3888000
maxVolumeDataSizeMB = 5976884
The following are my questions:
Which one do I have to use, maxVolumeDataSizeMB or maxTotalDataSizeMB inside the abc_transaction index.
If the customer wants to keep 90 days of abc_transaction data, as per the Splunk sizing link which I mentioned above the maxVolumeDataSizeMB = 11.4TB which is almost the disk size of the indexer, is it possible to set the entire disk size as maxVolumeDataSizeMB ?
Any help would be greatly appreciated.
Thanks
Bala.
... View more
