Cheap way to find min(_indextime) without walking all the indexes backwards?

[neiljpeterson]

I want to find the earliest _indextime by host for all my indexes. Specifically I need to know when the each forwarder first started indexing data... NOT the earliest timestamp of an event, but when the data itself was actually forwarded and indexed.

I can do this obviously

index=* | chart min(_indextime) as first by host | fieldformat "first"=strftime('first', "%c")

but it takes a while. Is there a way to do this more cheaply?

Here is an almost identical question: http://answers.splunk.com/answers/81718/search-for-oldest-event-in-splunk-by-_indextime-to-test-data... But unfortunately metadata and firsttime return min(_time) not min(_indextime) AFAIK

What is the easiest way to quickly grab the earliest _indextime by host without searching every event in every index?

[martin_mueller · Accepted solution]

This should be pretty brisk:

| tstats min(_indextime) where (index=* OR index=_*) by host

Run over all time, takes a lot less time than your example on my dev machine. Change the where if you're only looking for a specific index.

This search has completed and has returned 2 results by scanning 3,381,336 events in 4.096 seconds.

About a million EPS on a single machine with only one drive 😄
For comparison, the other way takes a bit longer, only about 60000 EPS:

index=* OR index=_* | stats min(_indextime) by host
This search has completed and has returned 2 result by scanning 3,381,718 events in 55.554 seconds.