Is there a way if an alert is generated with a transaction failure above threshold. and it should only throttle if the next alert result is less than the first transaction failures. if the next alert result is more than the previous alert result i should generate an alert, it shouldn't throttle.
for Example : alert is set for every 10 mins and the threshold for the failcount of APIs is 5 and we want to have a throttle for 60 mins.
events Time APIs failCount
1 00:10 "API-1" 10
2 00:20 "API-2" 5
3 00:30 "API-1" 7
4 00:40 "API-1" 15
first alert in : API-1 gets triggered for the first time. as its failcount is 10 which is above threshold(5). [ I want it throttle for 60 mins unless until if the upcoming results of API-1 failcount is above 10 ]
second Alert : API - 2 gets triggered as its on threshold.
third alert : I dont want anything to be generated because the API-1 is already triggered and its failcount is 7 which is below the results of first alert (failcount = 10) and also its throttle time period is 60 mins.
Fourth Alert : I want the alert to be generated as its failcount is 15 which more than the first alert (10).
is there a way that we can achieve this??
Its just an idea to control the Email Alerts.
... View more
I am trying to control the Email Alerts that are generated by splunk.
So once if a KPI's transaction failure is above the threshold and Alert is generated.
Is there a way, that a User reply's to that alert Email with a Keyword " STOP " or anyother word, and then that particular KPI in that Alert should have a throttle for some x mins.
... View more