Hi Team,
In my use-case, we are querying multiple hosts in a single query. Here we're using custom trigger conditions to trigger an alert. (e.g "search health='Unhealthy'")
The schedule is every 7 mins (Cron: */7 * * * *), for Time Range of the last 10 mins.
We need alert triggered only once per host till the alert is not cleared. (Unless another host is alerting. Not same host)
The purpose to do this is an automatic ticket generation. (If repetitive alerts are getting triggered, that will result in duplicate tickets generation)
If we use Throttle condition here, let's say throttle alerts for 1 hour, during that 1 hour time, any other host may start alerting, I won't be able to receive alert for that host as well.
Please suggest the best way to do this asap...
Can you change your query so it only counts unhealthy hosts if they weren't unhealthy in the previous time period? This assumes that the hosts that are unhealthy will continue to show up in your search until they are made healthy - consider using autoregress as one possible way of doing this