All Apps and Add-ons

Retention Limits on a index question



I am on a roll for questions today. Just learning Firebrigade here, cool app. But I don't understand how it's possible that I can have data in an index older than the max age of the index?

I have a 30 day max on my activemq index but we can see data 48 days old in there. Shouldn't splunk dump that? Is there a way to force it to dump?

alt text

0 Karma

Splunk Employee
Splunk Employee

The span of a bucket is defined as the time between the oldest event (in your case, 48 days) and the youngest event in that bucket. The span of the index in its entirety is the oldest event from any bucket to the youngest event in any bucket. A bucket's "age" is calculated only based upon the youngest event in the bucket. It's entirely possible (and this is what's happening in your case) that a bucket's data can span multiple days. Since the time-based expulsion of data buckets is based upon the bucket's age (and therefore the newest event in that bucket) this bucket isn't yet "old enough".

You can get a bit more info about this bucket with the "Retention > Bucket Age vs. Age Limit" dashboard. You'll see the ten oldest buckets (those next on the chopping block) including an indication of the oldest event in that bucket. If you don't see your March 18th dates in that list, then that bucket isn't yet old enough (and in fact may have future dates!).

As for the question of forcing the data to dump; not really. You're only at 8% of capacity on that index, so Splunk own built-in policies won't delete it. However, you could go under the hood and remove individual buckets if you wanted. Given the potential for "warranty-voiding" activity, I'm loath to provide more detail than that!

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.