How do you make an alert that doesn't trigger when condition goes true?

[abdulwsu]

I have an alert to send us an email every time an app on the server is stopped. I am searching for a stopped pattern and generating the alert. However, there are times when we manually stop this app for maintenance and start it back up after the work is done. Now, this would generate a false alert since the app is stopped.

Is there anyway to avoid this? I mean, i only want to get alerted when the app is stopped and not restarted.

Here's an example:

I stop the app for maintenance at 1.00 PM. A app is stopped error is logged in the logs. I start it back at 1.10.

After multiple different lines in the logs, starting flow message is logged. However, in my current alert, since it's stopped in logged, i get alerted even though I have started the app back up. I am trying to avoid that.

My logs when the app is stopped has this line(my current search pattern).

Cannot process event as "app" is stopped
Multiple lines of different texts and words.....
Starting flow: app

So I should only get alerted if the starting flow line is not present in the logs in the set time period.

[kmaron]

You might have to tweak this a bit depending on if you want to use host or not but maybe this will help.

index=foo sourcetype=bar ... ("Cannot process event as "app" is stopped" OR "Starting flow: app") 
 | streamstats count(eval(like(_raw, "Cannot process event as "app" is stopped") OR like(_raw, "Starting flow: app")) AS counts BY host
 | stats range(_time) AS downTimeSeconds count AS messages values(_time) as Time by host
 | eval now=now()
 | eval downTimeSeconds = if((messages<2), now - Time, downTimeSeconds)
 | where downTimeSeconds > 180