topic Re: How to trigger an alert when a metric is wrong on 3 consecutive spans? in Splunk Search

How to trigger an alert when a metric is wrong on 3 consecutive spans?

FBachelin — Fri, 01 Jul 2022 03:23:13 GMT

Hi everyone, I have a search on approval success rates :

stats count as TOTAL,count(eval(criteria)) as APPROVED | eval APPROVEDPERCENT=if(TOTAL>0,round((APPROVED*100)/TOTAL,2),100)

I would then get the percentage of approved transactions on the time range.

I would like to raise an alert when this approvedpercent is less than 50 in 3 consecutive span of 15min.

I have tried the following based on another post:
| timechart span=15min max(APPROVEDPERCENT) as APPPERCENT | where APPPERCENT<50 | stats count as NumberNOK

(that I could run with the alert trigger condition as NumberNOK>=3 on last 45min)

But I ran this search on a time range where approval is 100% and NumberNOK is null . Anyone can help on this search?

thank you in advance

Re: How to trigger an alert when a metric is wrong on 3 consecutive spans?

ITWhisperer — Fri, 01 Jul 2022 06:35:23 GMT

| timechart span=15m count as TOTAL count(eval(criteria)) as APPROVED | eval APPROVEDPERCENT=if(TOTAL>0,round((APPROVED*100)/TOTAL,2),100) | where APPROVEDPERCENT < 50 | stats count as NumberNOK

Re: How to trigger an alert when a metric is wrong on 3 consecutive spans?

FBachelin — Fri, 01 Jul 2022 16:25:40 GMT

Thank you so much, it is working . I just have now to trigger the alert if NumberNOK>=3. Thanks

Re: How to trigger an alert when a metric is wrong on 3 consecutive spans?

ITWhisperer — Fri, 01 Jul 2022 16:36:20 GMT

You could add a where command

| where NumberNOK >= 3

Then trigger the alert if there are any rows