Discrepancies between metadata and tstats latest time results

[merch_sf]

I'm trying to figure out which search will most accurately tell me when events with future timestamps are being detected.

Somebody on the team built this search:
| tstats min(_time) as earliest_time, max(_time) as latest_time by index
| eval daysOfLogs=round((latest_time - earliest_time)/60/60/24, 2)
| eval eventsInFuture=if(latest_time > now(), "yes", "no")
| eval tnow = now()
| convert ctime(*time)
| lookup index_to_env index
| convert ctime(tnow)

That search doesn't show any sourcetypes with future data.

This search, on the other hand, shows that multiple sourcetypes are showing future timestamps:
| metadata type=sourcetypes index=* index!=_* | eval now=now() | eval futuretime=lastTime-now | where futuretime>0

Based on what I've seen searching on the raw events with a "latest=+20d@d", the tstats command is the one that isn't seeing the future events...

Any idea what is causing this behavior?

[esix_splunk]

You can very easily add these to your tstats searches. For example..

| tstats min(_time) as earliest_time, max(_time) as latest_time WHERE earliest=-30d@d latest=+1y@y by index sourcetype
| eval daysOfLogs=round((latest_time - earliest_time)/60/60/24, 2)
| eval eventsInFuture=if(latest_time > now(), "yes", "no")
| eval tnow = now()
| convert ctime(*time)
| convert ctime(tnow)

You can very much add earliest and latest time boundaries in tstats with the WHERE clause, and you can update BY clause to include any of the indexed fields ( _time, source, sourcetype, host.) Additionally, tstats doesn't show the indexed time, it shows the parsed event time as searched for with min and max and the default behavior is to use the timerange picker's earliest and latest, which is why you wont see future events by default.

Try the above search in your environment.

[PavelP]

Hello @merch_sf

metadata and dbinspect return a timestamp of the latest event:

• dbinspect - The timestamp for the last event in the bucket, which is the time-edge of the bucket furthest towards the future.
  
• metadata - The lastTime field is the timestamp for the last time that the indexer saw an event

Edit: as @esix_splunk mentioned in the post below, this statement is wrong: tstats shows last _indextime, so it is not suitable for your purpose

[to4kawa]

see reference https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Tstats