monitor that proper amount of events are coming in per host

[jfraiberg]

I am trying to build an app that will set a baseline per host of event count that will alert me when a hosts event count has dropped by a certain % per hour.

Has anyone tried something like this? I find that monitoring the agent alone is not enough and would like to know if a host is sending less events than normal.

Any ideas or advice would be welcome 🙂

[ziegfried · Accepted solution]

This search would compare the count by host of the previous hour with the current hour and filter those where the count dropped by more than 10%:

earliest=-2h@h latest=@h | stats count by date_hour,host | stats first(count) as prev last(count) as cur by host | eval cur_pct=(cur/prev)*100 | where cur_pct<90

[jfraiberg]

This totally worked, thanks! The search itself took forever though and took alot of resources to run. I will play with it some more. thanks!!!

[jameswatts]

You can speed the search up significantly with tstats:
|tstats count by host, _time span=1h| stats first(count) as prev last(count) as cur by host | eval cur_pct=(cur/prev)*100 | where cur_pct<90