I am setting up alerts and facing few issues in this. I have a custom condition to be met for an alert to be triggered and this search runs every 5 mins. So if my custom condition is I>B where I is the count of number of incidents for a Workgroup and B is the threshold for that Workgroup it needs to send an email. And I set the throttling at 20 mins.
Now the issue is I need to make the alert come in a way that if an alert has been created for one WorkGroup then it shouldn't send out an alert for that workgroup till the data goes below the threshold. I am doing this because when an alert is received user will be going through the WG and he doesn't need another alert for the same WG again unless the WG's issue has been solved where it goes below threshold (I<B) and again it crosses the threshold. At the same time the search needs to generate alerts for other Workgroups which have crossed thresholds but follow the same mechanism as above.
Here's some more info on how to get the lookup method working, assume that we're working with the internal logs and we want to alert for a sourcetype when the high threshold is above 100 and want to throttle until the count drops below 10
//1. report search that that generates stats of interests index=_internal | stats count AS curr BY sourcetype //2. load and update the lookup "state" table | append [inputlookup throttle | rename curr AS prev] | stats first(curr) AS curr, first(prev) AS prev BY sourcetype | outputlookup throttle //3. apply your condition search | search curr>100 (prev="" OR prev<10)
When this search completes it will only output the results that you need need to alert for - so it is best to set your alert condition to "number of results" "greater than" 0
I will try this out. In the 1 statement do we need to put the stats in a file throttle ? I am not familiar with how append works, hence the question. will go through the documentation for append.
One way to solve this is through keeping state - ie you're not looking for time base throttling but for alert state based throttling. To achieve this you have to somehow keep state for each workgroup, there are two ways to do this: