_internal index data not archiving/deleting after 30 days.

[Robbie1194]

Hi guys,

I was wondering if anyone knows why my _internal index information is not archiving/deleting from frozen after 30 days

It wont let me attach a screenshot but in the DMC it shows that the "Data Age vs Frozen Data (days)" is 103/30... Which isn't right!

I can see that the value of frozenTimePeriodInSecs in system/default/indexes.conf is 2592000 (30 days) and using btool shows that the value is being taken but I don't know why it isn't working? Any ideas?

I was thinking of making a new app for config and change it to 31 days to see if it changes anything? Does anyone think this would work? I'm in a clustered environment so I'm a bit worried to make any changes in case it makes it worse!

Any help will be appreciated.

Cheers!

[mattymo]

check out the | dbinspect command to examine the buckets in the index. As Teunlaan commented, Splunk will only freeze a bucket once the LATEST event eclipses the frozenTimeInSecs.

You can use dbinspect and a little eval magic to convert the earliest and latest event time to confirm the timespan your buckets cover.

https://answers.splunk.com/answers/112500/dbinspect-fields-names-and-format-changed-in-6.html

If you have low traffic on the box, you will need to tweak indexes.conf for your _internal indexes to set maxHotSpanSecs to something like 86401, to roll it every day, or 604801 to force the bucket to close after 1 week.

Also, if somehow you have future timestamps, this can cause problems with rolling.

[teunlaan]

What is your bucket size?
It will only delete data if the last message in your bucket is older than 30 day's