topic Re: How to turn off an alert for 30 minutes on a given day? in Alerting

How to turn off an alert for 30 minutes on a given day?

cj039165 — Mon, 21 Nov 2016 17:13:20 GMT

Hello -

I have an alert that I want to 'suppress' / 'turn off' for 30 min a week. Every Sunday a connection is dropped from 2:45pm to 3:15pm. The drop is part of 'normal' Sunday work that occurs. We don't need the 'false positives' hitting our on-call. Is there a way to stop alerting for just 30 min on a given day?

Thanks,

Carl

Re: How to turn off an alert for 30 minutes on a given day?

jkat54 — Mon, 21 Nov 2016 18:01:15 GMT

You'd have to write a cron type schedule for that or possibly more than one cron schedule. All these assume you run every 15 minutes.

Something like

 */15 * * * 1,2,3,4,5

And another for Sunday normal hours.

 */15 0,1,4-23 * * 0

Then one for 2-230 and 315-4 on Sunday

 0,15,30 2 * * 0
 15,30,45 3 * * 0

But use the same search for all... Name them differently, etc.

Re: How to turn off an alert for 30 minutes on a given day?

SierraX — Mon, 21 Nov 2016 18:12:04 GMT

Hello,
I would embed it in a search:

With a search

 | search NOT dactivate=*

Kind Regards
SierraX

Re: How to turn off an alert for 30 minutes on a given day?

cj039165 — Tue, 29 Sep 2020 11:51:13 GMT

Thanks for the response SierraX. Here is the search I'm running. New to Splunk, I'm getting an error message "Error in 'eval' command: The expression is malformed. Expected"

index=hdx_was source="/hdx1/was70-32/AppServer/profiles/AppSrv01/logs/PRD3_XF*/SystemOut.log" "A connection failed but has been re-established" |eval wday=strftime(now(),"%a"),homi=strftime(now(),"%H%M"),dactivate=if(wday"Sun" AND homi>=1445 AND homi<=1530,"off",NULL)

Re: How to turn off an alert for 30 minutes on a given day?

SierraX — Mon, 21 Nov 2016 21:00:07 GMT

Sorry for the late response...
when this is a 1to1 copy of the search, you forgot a = (equal) between wday and "Sun"

Re: How to turn off an alert for 30 minutes on a given day?

cj039165 — Tue, 22 Nov 2016 14:18:09 GMT

Thanks for the help.

Re: How to turn off an alert for 30 minutes on a given day?

cj039165 — Tue, 29 Sep 2020 12:08:48 GMT

Hello -

Got back to working on this. For some reason this is still alerting between 14:45 and 16:00 on Sundays. Not sure what I'm missing. Thanks.

index=hdx_was source="/hdx1/was70-32/AppServer/profiles/AppSrv01/logs/PRD3_XF*/SystemOut.log" "A connection failed but has been re-established" | eval wday=strftime(now(),"%a"),homi=strftime(now(),"%H%M"),dactivate=if(wday="Sun" AND homi>=1445 AND homi<=1600,"off",NULL)