Deployment Architecture

how is age of an event calculated before the bucket is frozen

Explorer

hi, when we set frozenTimePeriodInSecs=30 days for an index. i have read through the documentation and forums that the bucket moves to frozen state when all the data in the given bucket is more than 30 days old. but i need some more clarity. basically the condition i think is one of the below for the latest event in the bucket.

_time > frozenTimePeriodInSecs
OR
_indextime >frozenTimePeriodInSecs

Please let me know which time controls the retention.

0 Karma
1 Solution

Super Champion
0 Karma

Super Champion
0 Karma

Explorer

Thanks Luke.. The explanation in wiki link was a bit complex and took time to understand but it certainly supports your answer.

I was running two parallel daily reports - one based on _time and the other based on _indextime to verify if my retention values were working as expected. But both confusing me.
Now i will concentrate on _time based one for my further analysis.

Thanks Again.

0 Karma

Super Champion

It is complex, but necessarily so because while you may want old data, you probably most certainly want current data.

What may not be evident in the wiki post is the indexing very old logs and logs 'in the future'.

Let's say for example you have logs that are older than the MAXDAYSAGO parameter from props.conf (default 2000 days) - all events older than the MAXDAYSAGO will have the time value of the last acceptable timestamp in that log file, and if all events are older than MAXDAYS_AGO, then all events will have the current (index time) timestamp for _time.

So, in addition to knowing that Freezing data is based on time, you must also understand that it is based on the most recent time in the buckets (restart Splunkd = new buckets), and that indexing very old logs may give you event times (time) that are not what you expected (MAXDAYSAGO defaults to 2000 days).

SplunkTrust
SplunkTrust

I'm 95% certain it's _time.

---
If this reply helps you, an upvote would be appreciated.
0 Karma