Deployment Architecture

What Should I Expect After Implementing A Retirement Policy?

gph12
Explorer

Hello,

I'm looking for advice\info on how retirement polices work in practice. Based on this document, I set a retirement policy for 1 index to start with to remove data older than 2 years. I set it to:
frozenTimePeriodInSecs = 63072000

What can I expect after doing this? I have seen the number of events in one index go down from 429 million to 421 million. But there are still events older than two years.

Is there a process or log that shows the retirement activity--such has how many events were removed on a particular day\week\month?

I presume the index itself will not be reduced in size, just the number of events. Is that correct?

If the index does not shrink, will new events fill up the white space made available by retired events? Or will the index continue to grow? (I have two conflicting goals--I don't want to run out of disk space but I have a compliance requirement to keep events for a certain period of time. Otherwise, I would set a maximum size on the indexes.

Thanks in advance.

0 Karma

somesoni2
Revered Legend

Data retirement policies doesn't work on per event basis, instead it works on data buckets for that index. It'll only delete, cold stage, buckets only when the latest event in that bucket is older than the set frozenTimePeriodInSecs . (say in a bucket you've data with _time ranging from 10/04/2015 to 11/18/2015, that bucket won't be deleted because the latest event on the bucket, 11/18/2015 is not older than 2 years from now, even though it contains other events which are older).

I would suggest a read of this to understand the retention policies better.
https://docs.splunk.com/Documentation/Splunk/7.0.0/Indexer/Setaretirementandarchivingpolicy

0 Karma

gph12
Explorer

Thanks for the comment. Yes, that's the document I tried to link but it didn't take for some reason.

Your explanation of buckets and the time ranges makes sense. Thanks again.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...