topic Re: Alert when count is less than a certain threshold in Alerting

Alert when count is less than a certain threshold

cpraznowski_spl — Mon, 09 May 2016 19:54:36 GMT

Hi...here is my search:

sourcetype="isc:dhcp"   earliest=-10m@s latest=now |    stats count  as dhcp_count by _time | where dhcp_count<5000

I'll usually get returned stats:
4800
10,000
11,000

I have this running on 5 minute cron schedule...don't see results. Perhaps I shall try real-time every 5 minutes...any thoughts appreciated

Re: Alert when count is less than a certain threshold

somesoni2 — Mon, 09 May 2016 20:15:47 GMT

Don't do real-time scheduled alert as it never ends. When you say your don't see result mean that alert is not triggered?

Re: Alert when count is less than a certain threshold

cpraznowski_spl — Mon, 09 May 2016 20:25:08 GMT

hi...yes, thanks for responding, I get the results above in my stats table, but when I access, Activity, Triggered alerts (search all)...I do not see results. In the above example, I was expecting to see the alert triggered b/c the count was 4,800...ie <5000

Re: Alert when count is less than a certain threshold

somesoni2 — Mon, 09 May 2016 20:48:02 GMT

The result that you gave, is it from the search which includes the where clause? Can you check in the scheduler logs to see what is the result_count returned by the search?

index=_internal sourcetype=scheduler savedsearch_name="YourAlertSearchName"

Re: Alert when count is less than a certain threshold

jkat54 — Tue, 10 May 2016 00:58:56 GMT

Why do you have the "by clause" in your stats command, or a stats command at all? Shouldnt it just be:

 sourcetype="isc:dhcp" earliest=-10m@s latest=now | where count<5000

See if that helps.

Re: Alert when count is less than a certain threshold

woodcock — Tue, 10 May 2016 04:57:34 GMT

You have not specified the most important part: what is your timepicker? Typically, this range is similar to the period so maybe you are running with Last 5 minutes which maybe is not a long enough timespan to ever rack up 5000 hits. When you say you "usually get ...", what was your timepicker value?

Re: Alert when count is less than a certain threshold

cpraznowski_spl — Tue, 10 May 2016 15:21:41 GMT

Last ten minutes: earliest=-10m@s latest=now

Re: Alert when count is less than a certain threshold

cpraznowski_spl — Tue, 10 May 2016 15:23:34 GMT

by clause because the dchp server sends logs every 5 minutes
12:10 4800
12:15 10000
12:20 12000

The span is 10 minutes in period, so I want to alert when any of the _time dimensions < 5000

Re: Alert when count is less than a certain threshold

cpraznowski_spl — Tue, 10 May 2016 15:24:33 GMT

cool, will take a look

Re: Alert when count is less than a certain threshold

woodcock — Tue, 10 May 2016 15:26:26 GMT

Surely you do not have 11K events per second, right? Your BY _time clause seems silly to me; probably you should be doing this instead:

 sourcetype="isc:dhcp" earliest=-10m@s latest=now | stats count AS dhcp_count | where dhcp_count<5000

And I would put the | where ... part inside your alert threshold settings.

Re: Alert when count is less than a certain threshold

cpraznowski_spl — Tue, 10 May 2016 15:31:43 GMT

Not quite....11K events per 5 minute period

..And I would put the | where ... part inside your alert threshold settings.... that's what I think I need to try..thanks

Re: Alert when count is less than a certain threshold

somesoni2 — Tue, 10 May 2016 15:40:50 GMT

Try this

alert search

sourcetype="isc:dhcp"   earliest=-11m@m latest=-1m@m|    stats count  as dhcp_count by _time | where dhcp_count<5000

Alert schedule - Cron

1-59/5 * * * *

Alert condition

When number of events > 0

Re: Alert when count is less than a certain threshold

cpraznowski_spl — Tue, 10 May 2016 15:50:57 GMT

woohoo, thanks a ton

Re: Alert when count is less than a certain threshold

cpraznowski_spl — Tue, 10 May 2016 15:51:45 GMT

thanks ! 🙂