Alerting

How to trigger alert based on result filed value count

Kwip
Contributor

I want to generate alert for the below query if it gives more than 10 results per minute for the continues 10 minutes. I can achieve it from the below query and setting the alert condition as search result equal to 10 and scheduling the alert to run on every 1 minute.

Query - eventtype=prod_servers sc_status!=200 | eval ComputerStatus=host."-"sc_status | bucket span=1m _time | stats count as TotalErrCount by _time ComputerStatus | where TotalErrCount >10 | stats value(ComputerStatus ) as ErrorCode count by _time

But the problem is the eval field ComputerStatus comprises of many values (I.e, server1-404, server2-404, server1-500, server3-403.,,,) so in 10 minutes span I am getting result like as below

02:27 10:01-server2-404
02:27 10:02-server2-404
02:27 10:03-server1-500
02:27 10:04-server1-500
02:27 10:05-server1-500
02:27 10:06-server1-500
02:27 10:07-server3-403
02:27 10:08-server3-403
02:27 10:09-server3-403
02:27 10:10-server2-404

My alert conditions are satisfied for the above results and alert getting triggered. but I want the alert to be generated if any one of the filed value count is greater than 10 per 1 minute for the continuous 10 minutes.
Example
02:27 10:01-server2-404 OR server1-500
02:27 10:02-server2-404 OR server1-500
02:27 10:03-server2-404 OR server1-500
02:27 10:04-server2-404 OR server1-500
02:27 10:05-server2-404 OR server1-500
02:27 10:06-server2-404 OR server1-500
02:27 10:07-server2-404 OR server1-500
02:27 10:08-server2-404 OR server1-500
02:27 10:09-server2-404 OR server1-500
02:27 10:10-server2-404 OR server1-500

Tags (1)
0 Karma
1 Solution

pradeepkumarg
Influencer

I replied on your other thread. Posting same here

eventtype=prd_servers sc_status!=300 sc_status!=200 sc_status!=0 | eval computerstatus=host:"-":sc_status | bin span=1m _time  | stats count by _time, computerstatus | search count>10 | eventstats count as occurences by computerstatus | search occurences > 10

View solution in original post

0 Karma

pradeepkumarg
Influencer

I replied on your other thread. Posting same here

eventtype=prd_servers sc_status!=300 sc_status!=200 sc_status!=0 | eval computerstatus=host:"-":sc_status | bin span=1m _time  | stats count by _time, computerstatus | search count>10 | eventstats count as occurences by computerstatus | search occurences > 10
0 Karma

Kwip
Contributor

Thank you!

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...