How do I setup an alert that triggers and take action on second search?

[antmob]

I have an alert that triggers when the search returns 0 events for the last couple of hours and sends a slack message. It runs every 5 minutes on cron and looks a few hour back. However, for some reason that I don't know, the alert false triggers some times when it should not and when I manually do the search for which it triggered, I see a bunch of events during that time span. This happens once a month or so.

So if anyone know a solution for this or why this happens that would be great. If not, I'm thinking of changing the alert so that it only triggers if the result is 0 for 2 searches in a row (5 minutes in between) to avoid the false triggers. Is it possible to do this and how?

[Nisha18789]

Hi @antmob 

For investigating the issue, you might want to start with checking the actual search window of the false alert instance using query like :

index=_internal sourcetype=scheduler status=success savedsearch_name=<name of your alert>

And, for 2nd question on 2 consecutive - 0 results, you can join the results of below query with your alert logic or you can use summary index to store the results and join with summary indexed data.

index=_internal sourcetype=scheduler status=success savedsearch_name=<name of your alert> result_count=0

Hope this helps!