Getting Data In

Search App - Earliest Event not Updated when events are archived

silvermail
Path Finder

Hello all,

Not sure if anyone has encountered this before, but I have events that are purged off but when I am in the Search App, the "Earliest Event" are still showing events that had been deleted. I noticed that the "Earliest Event" is actually based on a search that goes like this - "| stats min(firstTime) as min | convert ctime(min)"

So is it possible for this Earliest Event to reflect the timestamp of the earliest logs correctly?

Thanks!

Tags (1)
0 Karma
1 Solution

Lowell
Super Champion

I'm not 100% sure I'm following your question. I'm guessing that you are talking about deleting events (using the "| delete" command). Or are you talking about buckets that were aged out of your index due to your index retention policy? I'm guessing that your reference to 'firstTime', is referring to the the field being returned by a metadata command. (Such as | metadata index=main type=hosts).

It's possible for your metadata not be updated in some delete cases, but it seems like it often is updated correctly. The bucket rolling should certainly trigger an index-level metadata rebuild.

You can force the index-level metadata rebuild by creating an empty file in your indexe's "db" folder called meta.dirty. It should be deleted automatically whenever the metadata for the index has been rebuild.

Is this causing you a problem?

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee
0 Karma

silvermail
Path Finder

Thanks Lowell! I was actually referring to the buckets that were aged out to to the index retention policy. I will try to do a index-level metadata rebuild and see how it goes.

0 Karma

Lowell
Super Champion

I'm not 100% sure I'm following your question. I'm guessing that you are talking about deleting events (using the "| delete" command). Or are you talking about buckets that were aged out of your index due to your index retention policy? I'm guessing that your reference to 'firstTime', is referring to the the field being returned by a metadata command. (Such as | metadata index=main type=hosts).

It's possible for your metadata not be updated in some delete cases, but it seems like it often is updated correctly. The bucket rolling should certainly trigger an index-level metadata rebuild.

You can force the index-level metadata rebuild by creating an empty file in your indexe's "db" folder called meta.dirty. It should be deleted automatically whenever the metadata for the index has been rebuild.

Is this causing you a problem?

Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...