topic How to troubleshoot why 90 data data retention configuration is not being applied? in Getting Data In

How to troubleshoot why 90 data data retention configuration is not being applied?

rblalock — Mon, 17 Nov 2014 20:07:42 GMT

I want to freeze all data older than 90 days.

My /opt/splunk/etc/system/local/indexes.conf file looks like this

[default]

[_audit]

[main]
rotatePeriodInSecs = 60
coldToFrozenDir = /logs/frozen
maxTotalDataSizeMB = 400000
frozenTimePeriodInSecs = 7776000

But I still have data that is up to six months old. Can someone suggest other places to look to correct this?

Re: How to troubleshoot why 90 data data retention configuration is not being applied?

martin_mueller — Mon, 17 Nov 2014 22:04:17 GMT

First, verify the value stuck by running splunk cmd btool indexes list main.

Then, check if the bucket containing that old data happens to also contain newer data. It'll get rolled when the youngest event is older than the frozen period.

Re: How to troubleshoot why 90 data data retention configuration is not being applied?

somesoni2 — Mon, 17 Nov 2014 23:56:44 GMT

See below link which explains various properties involved in setting up the retention policy and bucket life cycle.

http://wiki.splunk.com/Deploy:BucketRotationAndRetention

Basically, you need to setup maxHotSpanSecs and maxWarmDBCount to values so that your data bucket is getting rolled over to frozen. It will be deleted only after it's moved to frozen state.