Splunk Dev

how to set 3 month searchable data retention on a high volume per day index, currently only seeing 2 weeks worth?

u788332
New Member

maxDataSize if set to auto the default is 750 MB and 10GB on 64 bit and 1GB on 32 bit hosts, if it is set to auto_high_volume, is that correct?

frozenTimePeriodInSecs = 8640000
maxWarmDBCount = 50
maxHotBuckets = 55
maxDataSize = auto
So going by the configuration for this index , the calculations would come to around (50+55)*750MB = 76GB, also even if we go by 10 GB it would come to (50+55)*10GB = 1050GB for searchable data.
monthly usage for this index, equates to 2906.571GB, if we go by (default 10GB bucket size) then searchable data would be just 1050 GB that does fit the observed two week retention.
how much would we need to increase the maxWarmDBCount to? To allow for Data to be searchable still for 100 days?

thanks in advance

Tags (1)
0 Karma

jkat54
SplunkTrust
SplunkTrust

Your fronzenTimePeriodInSecs is 100 days... 86400 (secs/day) * 100 = 8640000

Try this for 30 days retention:

Add 1 sec to avoid ohSnap

[indexName]
maxHotSpanSecs = 86401 #not 86400 to avoid ohSnap
maxHotIdleSecs = 86401
frozenTimePeriodInSecs = 2592000
...

This will make sure the hot buckets close at the end of a day instead of waiting for the hot buckets to fill before they roll.

The net effect is buckets from 31 days ago are rolled to frozen.

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...