Alerting

Alert when count is less than a certain threshold

cpraznowski_spl
Splunk Employee
Splunk Employee

Hi...here is my search:

sourcetype="isc:dhcp"   earliest=-10m@s latest=now |    stats count  as dhcp_count by _time | where dhcp_count<5000

I'll usually get returned stats:
4800
10,000
11,000

I have this running on 5 minute cron schedule...don't see results. Perhaps I shall try real-time every 5 minutes...any thoughts appreciated

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Try this

alert search

sourcetype="isc:dhcp"   earliest=-11m@m latest=-1m@m|    stats count  as dhcp_count by _time | where dhcp_count<5000

Alert schedule - Cron

1-59/5 * * * *

Alert condition

When number of events > 0

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try this

alert search

sourcetype="isc:dhcp"   earliest=-11m@m latest=-1m@m|    stats count  as dhcp_count by _time | where dhcp_count<5000

Alert schedule - Cron

1-59/5 * * * *

Alert condition

When number of events > 0
0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

woohoo, thanks a ton

0 Karma

woodcock
Esteemed Legend

You have not specified the most important part: what is your timepicker? Typically, this range is similar to the period so maybe you are running with Last 5 minutes which maybe is not a long enough timespan to ever rack up 5000 hits. When you say you "usually get ...", what was your timepicker value?

0 Karma

woodcock
Esteemed Legend

Surely you do not have 11K events per second, right? Your BY _time clause seems silly to me; probably you should be doing this instead:

 sourcetype="isc:dhcp" earliest=-10m@s latest=now | stats count AS dhcp_count | where dhcp_count<5000

And I would put the | where ... part inside your alert threshold settings.

0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

Not quite....11K events per 5 minute period

..And I would put the | where ... part inside your alert threshold settings.... that's what I think I need to try..thanks

0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

thanks ! 🙂

0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

Last ten minutes: earliest=-10m@s latest=now

0 Karma

jkat54
SplunkTrust
SplunkTrust

Why do you have the "by clause" in your stats command, or a stats command at all? Shouldnt it just be:

 sourcetype="isc:dhcp" earliest=-10m@s latest=now | where count<5000

See if that helps.

0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

by clause because the dchp server sends logs every 5 minutes
12:10 4800
12:15 10000
12:20 12000

The span is 10 minutes in period, so I want to alert when any of the _time dimensions < 5000

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Don't do real-time scheduled alert as it never ends. When you say your don't see result mean that alert is not triggered?

0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

hi...yes, thanks for responding, I get the results above in my stats table, but when I access, Activity, Triggered alerts (search all)...I do not see results. In the above example, I was expecting to see the alert triggered b/c the count was 4,800...ie <5000

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The result that you gave, is it from the search which includes the where clause? Can you check in the scheduler logs to see what is the result_count returned by the search?

index=_internal sourcetype=scheduler savedsearch_name="YourAlertSearchName"  
0 Karma

cpraznowski_spl
Splunk Employee
Splunk Employee

cool, will take a look

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...