Hi,
This search string have helped us a lot during investigation. It paints a timechart / graphicall representation of when there was a dip in ingestion together with the log's raw time. Since it's using tsats, it's also fast.
| tstats min(_indextime) as it count where index=my_index by _time span=1ms
| eval _time = it
| fields - it
| timechart span=5m sum(count) as it
| join _time
[| tstats count as raw where index=my_index by _time span=5m ]
| fillnull value=0
Here's how the graph looks like (set linechart Y to Log)
As you can see, the blue line dipped (index-time) and yellow line (log's raw time) shows that logs that did not ingest during the time the index-time dipped have already caught up.
Since this search is using min(_indextime) and is joining it in _time, there's no guarantee that it's 100% accurate. However, I can confidently say that it's almost always reliable.
If the two lines are overlapping with oneother (appearing almost as though there's only one line), that means that the logs are ingested almost instantaneously.
