Deployment Architecture

how is age of an event calculated before the bucket is frozen

pavankumarh
Path Finder

hi, when we set frozenTimePeriodInSecs=30 days for an index. i have read through the documentation and forums that the bucket moves to frozen state when all the data in the given bucket is more than 30 days old. but i need some more clarity. basically the condition i think is one of the below for the latest event in the bucket.

_time > frozenTimePeriodInSecs
OR
_indextime >frozenTimePeriodInSecs

Please let me know which time controls the retention.

0 Karma
1 Solution

lukejadamec
Super Champion
0 Karma

lukejadamec
Super Champion

It's _time
See this post. It provides a very good review:
https://wiki.splunk.com/Deploy:SplunkBucketRetentionTimestampsAndYou

0 Karma

pavankumarh
Path Finder

Thanks Luke.. The explanation in wiki link was a bit complex and took time to understand but it certainly supports your answer.

I was running two parallel daily reports - one based on _time and the other based on _indextime to verify if my retention values were working as expected. But both confusing me.
Now i will concentrate on _time based one for my further analysis.

Thanks Again.

0 Karma

lukejadamec
Super Champion

It is complex, but necessarily so because while you may want old data, you probably most certainly want current data.

What may not be evident in the wiki post is the indexing very old logs and logs 'in the future'.

Let's say for example you have logs that are older than the MAX_DAYS_AGO parameter from props.conf (default 2000 days) - all events older than the MAX_DAYS_AGO will have the _time value of the last acceptable timestamp in that log file, and if all events are older than MAX_DAYS_AGO, then all events will have the current (index time) timestamp for _time.

So, in addition to knowing that Freezing data is based on _time, you must also understand that it is based on the most recent time in the buckets (restart Splunkd = new buckets), and that indexing very old logs may give you event times (_time) that are not what you expected (MAX_DAYS_AGO defaults to 2000 days).

richgalloway
SplunkTrust
SplunkTrust

I'm 95% certain it's _time.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Introduction to Splunk AI

How are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. Lucky for ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Maximizing the Value of Splunk ES 8.x

Splunk Enterprise Security (ES) continues to be a leader in the Gartner Magic Quadrant, reflecting its pivotal ...