Alerting

How to send alert for consecutive field values failure

manish_singh_77
Builder

Hi All,

I have a table where we have job_result field, whenever job_result field has "failure" value for consecutive 5 times then we need to set up an alert.

Example

job job _name time job result
abc abc_job 14:20 Failure
def def_job 15:30 Failure
ghi ghi_job 12:22 Failure
annc annc_job 14:20 Failure
defff defff_job 15:30 Failure
ghhhi ghhhi_job 12:22 Failure

Tags (2)
0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval _raw="job   job_name  time job_result
abc   abc_job   14:20 Failure
def   def_job   15:30 Failure
ghi   ghi_job   12:22 Failure
annc  annc_job  14:20 Failure
defff defff_job 15:30 Failure
ghhhi ghhhi_job 12:22 Failure
abc   abc_job   14:20 Success
def   def_job   15:30 Failure
ghi   ghi_job   12:22 Failure
annc  annc_job  14:20 Failure
defff defff_job 15:30 Failure
ghhhi ghhhi_job 12:22 Failure"
| multikv forceheader=1 
| table job job_name time job_result
| streamstats reset_on_change=t count(eval(job_result="Failure")) as fail_count by job_result
| eventstats max(fail_count) as alert_count

Hi, @manish_singh_777
this is sample code.

your_search
| table job job_name time job_result
| streamstats reset_on_change=t count(eval(job_result="Failure")) as fail_count by job_result
| eventstats max(fail_count) as alert_count

alert_count > 5, fire alert.

manish_singh_77
Builder

Hi @to4kawa

I want to send an alert if specific job is failing for 5 consecutive times.

0 Karma

to4kawa
Ultra Champion
your_search
 | table job job_name time job_result
 | streamstats reset_on_change=t count(eval(job="abc" AND job_result="Failure")) as fail_count by job_result
 | eventstats max(fail_count) as alert_count

please add job.

0 Karma
Get Updates on the Splunk Community!

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...

What’s New in Splunk Observability – September 2025

What's NewWe are excited to announce the latest enhancements to Splunk Observability, designed to help ITOps ...

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...