topic Re: Search App - Earliest Event not Updated when events are archived in Getting Data In

Search App - Earliest Event not Updated when events are archived

silvermail — Wed, 04 Aug 2010 22:26:59 GMT

Hello all,

Not sure if anyone has encountered this before, but I have events that are purged off but when I am in the Search App, the "Earliest Event" are still showing events that had been deleted. I noticed that the "Earliest Event" is actually based on a search that goes like this - "| stats min(firstTime) as min | convert ctime(min)"

So is it possible for this Earliest Event to reflect the timestamp of the earliest logs correctly?

Thanks!

Re: Search App - Earliest Event not Updated when events are archived

Lowell — Thu, 05 Aug 2010 01:02:55 GMT

I'm not 100% sure I'm following your question. I'm guessing that you are talking about deleting events (using the "| delete" command). Or are you talking about buckets that were aged out of your index due to your index retention policy? I'm guessing that your reference to 'firstTime', is referring to the the field being returned by a metadata command. (Such as | metadata index=main type=hosts).

It's possible for your metadata not be updated in some delete cases, but it seems like it often is updated correctly. The bucket rolling should certainly trigger an index-level metadata rebuild.

You can force the index-level metadata rebuild by creating an empty file in your indexe's "db" folder called meta.dirty. It should be deleted automatically whenever the metadata for the index has been rebuild.

Is this causing you a problem?

Re: Search App - Earliest Event not Updated when events are archived

silvermail — Thu, 05 Aug 2010 06:49:18 GMT

Thanks Lowell! I was actually referring to the buckets that were aged out to to the index retention policy. I will try to do a index-level metadata rebuild and see how it goes.

Re: Search App - Earliest Event not Updated when events are archived

Stephen_Sorkin — Thu, 19 Aug 2010 07:54:49 GMT

This appears to be a duplicate of: http://answers.splunk.com/questions/5321/indexes-data-retirement-earliest-event