topic Re: Firewall indexing latency: How to track when syslog is read by file monitor when the log is seen in search GUI? in Getting Data In

Firewall indexing latency: How to track when syslog is read by file monitor when the log is seen in search GUI?

hartfoml — Wed, 03 Sep 2014 18:20:56 GMT

I can find the latency for firewall log indexing like this

index=Firewall | eval diff_sec=(_indextime - _time)| where diff_sec > 0 | stats avg(diff_sec) as latency

this shows a latency of about 3 minutes

the problem is that when I search the logs I often do not get any messages or logs for the first 15 to 20 minutes.

is there a way I can track when the syslog is read by file monitor when the log is seen in the search GUI.

Re: Firewall indexing latency: How to track when syslog is read by file monitor when the log is seen in search GUI?

kristian_kolb — Wed, 03 Sep 2014 19:19:46 GMT

index=firewall | head 1 | eval diff=now()-_time | table diff

would work if you can assume that the logs are written to file, and that the [monitor] stanza picks them up, in a timely manner.

Basically, it will show the difference between when the search is run (now()) and the timestamp of the last event. This assumes that clocks are not too far off 🙂

Re: Firewall indexing latency: How to track when syslog is read by file monitor when the log is seen in search GUI?

hartfoml — Thu, 04 Sep 2014 13:41:42 GMT

Kristian, thanks for your suggestion. I used it to build this graph

I had a problem running this in real time but if I schedule for every 5 minutes and send results to summary I can get a fairly true history of latency. If you would like to put your comment in as an answer I will accept it.

Re: Firewall indexing latency: How to track when syslog is read by file monitor when the log is seen in search GUI?

MuS — Thu, 04 Sep 2014 13:49:52 GMT

here you go 🙂