Firewall indexing latency: How to track when syslog is read by file monitor when the log is seen in search GUI?

[hartfoml]

I can find the latency for firewall log indexing like this

index=Firewall | eval diff_sec=(_indextime - _time)| where diff_sec > 0 | stats avg(diff_sec) as latency

this shows a latency of about 3 minutes

the problem is that when I search the logs I often do not get any messages or logs for the first 15 to 20 minutes.

is there a way I can track when the syslog is read by file monitor when the log is seen in the search GUI.

[kristian_kolb · Accepted solution]

index=firewall | head 1 | eval diff=now()-_time | table diff

would work if you can assume that the logs are written to file, and that the [monitor] stanza picks them up, in a timely manner.

Basically, it will show the difference between when the search is run (now()) and the timestamp of the last event. This assumes that clocks are not too far off 🙂

[MuS]

here you go 🙂

[hartfoml]

Kristian, thanks for your suggestion. I used it to build this graph

index=firewall | head 1 | eval diff=now()-_time | table diff | eval diffmin=diff/60 | gauge diffmin 0 5 10 15 20

I had a problem running this in real time but if I schedule for every 5 minutes and send results to summary I can get a fairly true history of latency. If you would like to put your comment in as an answer I will accept it.