Getting Data In

Search App - Earliest Event not Updated when events are archived

silvermail
Path Finder

Hello all,

Not sure if anyone has encountered this before, but I have events that are purged off but when I am in the Search App, the "Earliest Event" are still showing events that had been deleted. I noticed that the "Earliest Event" is actually based on a search that goes like this - "| stats min(firstTime) as min | convert ctime(min)"

So is it possible for this Earliest Event to reflect the timestamp of the earliest logs correctly?

Thanks!

Tags (1)
0 Karma
1 Solution

Lowell
Super Champion

I'm not 100% sure I'm following your question. I'm guessing that you are talking about deleting events (using the "| delete" command). Or are you talking about buckets that were aged out of your index due to your index retention policy? I'm guessing that your reference to 'firstTime', is referring to the the field being returned by a metadata command. (Such as | metadata index=main type=hosts).

It's possible for your metadata not be updated in some delete cases, but it seems like it often is updated correctly. The bucket rolling should certainly trigger an index-level metadata rebuild.

You can force the index-level metadata rebuild by creating an empty file in your indexe's "db" folder called meta.dirty. It should be deleted automatically whenever the metadata for the index has been rebuild.

Is this causing you a problem?

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee
0 Karma

silvermail
Path Finder

Thanks Lowell! I was actually referring to the buckets that were aged out to to the index retention policy. I will try to do a index-level metadata rebuild and see how it goes.

0 Karma

Lowell
Super Champion

I'm not 100% sure I'm following your question. I'm guessing that you are talking about deleting events (using the "| delete" command). Or are you talking about buckets that were aged out of your index due to your index retention policy? I'm guessing that your reference to 'firstTime', is referring to the the field being returned by a metadata command. (Such as | metadata index=main type=hosts).

It's possible for your metadata not be updated in some delete cases, but it seems like it often is updated correctly. The bucket rolling should certainly trigger an index-level metadata rebuild.

You can force the index-level metadata rebuild by creating an empty file in your indexe's "db" folder called meta.dirty. It should be deleted automatically whenever the metadata for the index has been rebuild.

Is this causing you a problem?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...