Indexes + data retirement + Earliest Event

[skippylou]

So looking at the Indexes page in Manager, I can tell that one of my indexes has hit the size limit and is successfully retiring/deleting data as necessary to stay under the size limit I set for it. However, the 'Earliest Event' timestamp listed is for the earliest that has ever been in the index, not what is actually in the index currently.

Any (easy) way to have this always show the actual 'Earliest Event' based on what is actually in the index at that time?

Thanks,

Scott

[Stephen_Sorkin · Accepted solution]

Excellent question.

This is because of the process for updating the metadata when a bucket is retired. It's relatively easy to decrement the counts for sources, sourcetypes and hosts when a bucket goes away. However, it's not efficient to update the earliest and latest timestamps, which are used for this display.

You have a couple of choices to fix this. You can either get at the data a different way, by means of the dbinspect command, or you could update the global {Hosts,Sources,SourceTypes}.data.

To retrieve the accurate time bounds with dbinspect, run the search:

| dbinspect index=<index_name> | convert timeformat="%m/%d/%Y:%T" mktime(earliestTime) mktime(latestTime) | stats min(earliestTime) as earliestTime max(latestTime) as latestTime | convert ctime(earliestTime) ctime(latestTime)

To update the metadata itself, you can create a meta.dirty file to cause the metadata to be regenerated:

touch $SPLUNK_HOME/var/lib/splunk/<index_name>/db/meta.dirty