Hello,
We have the "Opsec Lea for Checkpoint Linux" app pulling logs from the Checkpoint Enterprise log collector. However, the data is very slowly catching up to present and current data is several hours behind.
To see/visualize the delay, I can do a search like this with Realtime|AllTime in the Timepicker.
index=firewall | eval timeDiff=_time-_indextime | eval _time=now() | timechart limit=0 span=5m avg(timeDiff) by host
This time, shows me live events as they come in, and calculates the difference between the event time and Index time.
This is perfect for an ad-hoc search, but I would like to schedule something similar to run every few minutes, and dump the results to a lookup via outputlookup. The goal, is to monitor this data over a long period of time quickly, without re-running the above search over hours/days of data.
The problem is, you can't schedule this search as Realtime|AllTime since then the search will never complete. If you ran it for say, "Last XX Minutes" then it looks at _time and these events haven't happened yet since they are several hours behind. If we ran the search to look back several hours, to ensure we would see the events, then this skews my results as something like avg(timeDiff) would be over a larger block of time and isn't correct. Plus, if the amount of lag diminishes over time, querying the extra x-hours before the last event is unnecessary work.
Any thoughts?
Thanks In Advance,
Sean
