frozenTimePeriodInSecs only takes effect on IDX restart

[MrLR_02]

Hello,

I have defined a frozenTimePeriodInSecs for 1 hour on my IDX for a certain index, so that the logs it contains are only kept for 1 hour.
The definition of the frozenTimePeriodInSecs was made in the indexes.conf in the system/local directory
The problem I have, however, is that the frozenTimePeriodInSecs config only takes effect once when the IDX is restarted. Otherwise, the logs remain in this index for the defined retention period.

Has anyone already had the same problem and can help me with this?

Thanks in advance.

[PickleRick]

As an addition to what @livehybrid already said, see the .conf presentation https://conf.splunk.com/files/2017/slides/splunk-data-life-cycle-determining-when-and-where-to-roll-...

frozenTimePeriodInSecs only affects cold buckets. So a bucket has to first reach this stage in its life cycle. And hot buckets are rolled on a completely different basis than time-based retention limit.

That's it.

That's also why the usual questions like "how to make sure we have 2 days of hot buckets, a week of warm buckets and two months of cold buckets" get the response of "you can't do it this way".

[livehybrid · Accepted solution]

@MrLR_02 , the 1-hourfrozenTimePeriodInSecs will not affect buckets which are "hot" - ie they are actively open and being written to. If your buckets aren’t rolling from hot → warm → cold within an hour, retention will appear longer. The reason a restart causes them to roll to frozen is that the indexer closes the hot bucket when it restarts and thus becomes warm, and can then be frozen out.

To enforce deletion 1 hour after ingestion, you may need to review some of the following settings, ive included some examples below:

Force hot buckets to roll faster by setting:
Its worth understanding these and configuring as required - check https://docs.splunk.com/Documentation/Splunk/latest/Admin/Indexesconf#:~:text=maxHotSpanSecs%20%3D%2...for more info.

[your_index]
maxHotSpanSecs = 3600 # Hot bucket rolls to warm after 1h
maxHotIdleSecs = 60 # Rolls if idle for 1min
maxDataSize = auto_high_volume # Or lower to cap hot-bucket size

These ensure hot buckets roll to warm based on time, not just size.

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[MrLR_02]

Would these settings also have to be made if I set the retention period for this index to 1 day or possibly 1 week?