Getting Data In

What counts for Splunk retention time if events come in with a broken timestamp?

g_paternicola
Path Finder

if I have a retention at an index of 90days, but events come in with a broken timestamp that says 2018 or something; what counts for retention now? Indexed time or _time from the event?

Labels (5)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @g_paternicola,

as you can read at https://docs.splunk.com/Documentation/Splunk/8.2.6/Indexer/Setaretirementandarchivingpolicy, retention is managed at bucket level,

in other words, until in a bucket there's an event that is still in the retention period the bucked is maintained.

So if you have, in one bucket, events of 2018 and events of may 2022 they will be mainained until the newest events exceed the retention period.

This usually happends when you have few data in one index that you indexed in the same period.

You have only to wait.

Ciao.

Giuseppe

View solution in original post

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

There are many parameters which define how old events there could be in bucket. When you have set retention time to 90d it means that splunk start counting age of events on that bucket by _time. 90d means that splunk remove that bucket after all events in that bucket are older than 90d, it's not counting/freezing  those events one by one. So that's reason why there are some events much older than 90d. Anyway the volume and index sizes can lead situations when you haven't any events with 90d retention time. If there is no space then splunk removes longer buckets also. There are many answers which cover this situation more detail.

r. Ismo

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @g_paternicola,

as you can read at https://docs.splunk.com/Documentation/Splunk/8.2.6/Indexer/Setaretirementandarchivingpolicy, retention is managed at bucket level,

in other words, until in a bucket there's an event that is still in the retention period the bucked is maintained.

So if you have, in one bucket, events of 2018 and events of may 2022 they will be mainained until the newest events exceed the retention period.

This usually happends when you have few data in one index that you indexed in the same period.

You have only to wait.

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @g_paternicola,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the Contributors 😉

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...