Splunk Search

I'm looking to compare the _indextime to the _time field to look for anomalies

crlunde
Loves-to-Learn Everything

I'm looking to do some alerting or analysis to help troubleshoot lag time and logging. I'd like to compare the _indextime and _time fields to see how long it's taking the actual events to get indexed by Splunk. We have some users for 1 specific index that are stating they are seeing at least a couple of hours lag time between the event being generated and when Splunk is indexing the event. This is for initial research for the issue to help determine network issue, Splunk issue or other.

Thanks for any help!

Labels (4)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @crlunde,

let me understand: do you want a simple searcj to find the difference between _time and _indextime?

If yes, try something like this:

index=your index
| eval diff=_indextime-_time, indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S.%3N") 
| where diff>3
| table _time indextime diff

in my sample I filtered to take only the events with a difference o 3 seconds between _time and _indextime, obviously you can use the threshold you like.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...