I'm looking to compare the _indextime to the _time field to look for anomalies

[crlunde]

I'm looking to do some alerting or analysis to help troubleshoot lag time and logging. I'd like to compare the _indextime and _time fields to see how long it's taking the actual events to get indexed by Splunk. We have some users for 1 specific index that are stating they are seeing at least a couple of hours lag time between the event being generated and when Splunk is indexing the event. This is for initial research for the issue to help determine network issue, Splunk issue or other.

Thanks for any help!

[gcusello]

Hi @crlunde,

let me understand: do you want a simple searcj to find the difference between _time and _indextime?

If yes, try something like this:

index=your index
| eval diff=_indextime-_time, indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S.%3N") 
| where diff>3
| table _time indextime diff

in my sample I filtered to take only the events with a difference o 3 seconds between _time and _indextime, obviously you can use the threshold you like.

Ciao.

Giuseppe