Getting Data In

_internal index data not archiving/deleting after 30 days.

Robbie1194
Communicator

Hi guys,

I was wondering if anyone knows why my _internal index information is not archiving/deleting from frozen after 30 days

It wont let me attach a screenshot but in the DMC it shows that the "Data Age vs Frozen Data (days)" is 103/30... Which isn't right!

I can see that the value of frozenTimePeriodInSecs in system/default/indexes.conf is 2592000 (30 days) and using btool shows that the value is being taken but I don't know why it isn't working? Any ideas?

I was thinking of making a new app for config and change it to 31 days to see if it changes anything? Does anyone think this would work? I'm in a clustered environment so I'm a bit worried to make any changes in case it makes it worse!

Any help will be appreciated.

Cheers!

0 Karma

mattymo
Splunk Employee
Splunk Employee

check out the | dbinspect command to examine the buckets in the index. As Teunlaan commented, Splunk will only freeze a bucket once the LATEST event eclipses the frozenTimeInSecs.

You can use dbinspect and a little eval magic to convert the earliest and latest event time to confirm the timespan your buckets cover.

https://answers.splunk.com/answers/112500/dbinspect-fields-names-and-format-changed-in-6.html

If you have low traffic on the box, you will need to tweak indexes.conf for your _internal indexes to set maxHotSpanSecs to something like 86401, to roll it every day, or 604801 to force the bucket to close after 1 week.

Also, if somehow you have future timestamps, this can cause problems with rolling.

teunlaan
Contributor

What is your bucket size?
It will only delete data if the last message in your bucket is older than 30 day's

Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!