Alert throttle only when the Transaction failure is below from first Alert

[tarunchandra_re]

Hi,
Is there a way if an alert is generated with a transaction failure above threshold. and it should only throttle if the next alert result is less than the first transaction failures. if the next alert result is more than the previous alert result i should generate an alert, it shouldn't throttle.

for Example : alert is set for every 10 mins and the threshold for the failcount of APIs is 5 and we want to have a throttle for 60 mins.

 events  Time     APIs    failCount

   1     00:10   "API-1"   10
   2     00:20   "API-2"   5
   3     00:30   "API-1"   7
   4     00:40   "API-1"   15

first alert in : API-1 gets triggered for the first time. as its failcount is 10 which is above threshold(5). [ I want it throttle for 60 mins unless until if the upcoming results of API-1 failcount is above 10 ]

second Alert : API - 2 gets triggered as its on threshold.

third alert : I dont want anything to be generated because the API-1 is already triggered and its failcount is 7 which is below the results of first alert (failcount = 10) and also its throttle time period is 60 mins.

Fourth Alert : I want the alert to be generated as its failcount is 15 which more than the first alert (10).

is there a way that we can achieve this??

Its just an idea to control the Email Alerts.

[techiesid]

Hi,

With existing alert actions you wont be able to achieve this. Also here I can see you need to implement custom throttling logic. I think the best way here is to create custom alert actions. There basically you can write this whole logic in python including the custom throttling logic.
you can refer the below links for custom alert action,

https://docs.splunk.com/Documentation/Splunk/7.3.1/AdvancedDev/ModAlertsIntro

I also created some contents on custom alert action, you can refer them too,

https://youtu.be/UqJAc7rpFmQ
https://youtu.be/ZvzTowF9v9I
https://youtu.be/OT11XMB8Bu0

Sid