topic Re: Why alert is not triggering on exact time in Alerting

Why alert is not triggering on exact time

chandana204 — Mon, 16 Jul 2018 19:41:53 GMT

Hi,

From the past one week I have been looking into my alert jobs. I found that alerts are triggering 4 minutes before from the actual trigger time. Because of this time change i missed lot of alerts. May I know the reason, why Splunk Enterprice is considering actual time?

I am attaching screenshots for more understanding
The below image shows my alert trigger tim

The below image shows the alert triggered time. Based on trigger condition it should get trigger today at 12:00 AM but it got triggered yesterday 11:56 PM.

Please explain me if anyone know the reason behind this issue.

Thanks in Advance,
Chandana

Re: Why alert is not triggering on exact time

CarsonZa — Mon, 16 Jul 2018 20:15:08 GMT

we need more information.
whats the alert condition?
how are you saving that file?

Re: Why alert is not triggering on exact time

chandana204 — Mon, 16 Jul 2018 20:58:13 GMT

it's a scheduled alert. It'll trigger every Monday at 12:00 AM. The output file save as PDF.

I have been using this alert from past one and half month. It was triggering on exact time as specified trigger time. But from the last week it's getting trigger as mentioned above.

Re: Why alert is not triggering on exact time

pruthvikrishnap — Tue, 17 Jul 2018 00:27:40 GMT

Hi Chandana,
are there any other trigger conditions mentioned?

Re: Why alert is not triggering on exact time

chandana204 — Tue, 17 Jul 2018 16:40:57 GMT

nope, it'll trigger based on number of results.

Re: Why alert is not triggering on exact time

DalJeanis — Thu, 09 Aug 2018 19:13:37 GMT

I believe you are misinterpreting that time stamp. Just because the search was "created" shortly before the scheduled time does not mean that it ran early.

If you add this code to the end of the alert, you will see the actual time range that is covered.

| append [| makeresults | addinfo
   | eval Time = strftime(_time,"%Y-%m-%d %H:%M:%S")
   | eval Info_min_time=strftime(info_min_time,"%Y-%m-%d %H:%M:%S") 
   | eval Info_max_time=strftime(info_max_time,"%Y-%m-%d %H:%M:%S") 
   | eval Info_search_time=strftime(info_search_time,"%Y-%m-%d %H:%M:%S") 
   | eval Now=strftime(now(),"%Y-%m-%d %H:%M:%S")
   | table Time Now Info_min_time Info_max_time Info_search_time
   ]

info_min_time and info_max_time are the time bounds for events selected by the search. Info_search_time is the time the search was created. Now is the time the search started. Time is the time the makeresults command generated its output event, which is roughly a second after now().

More likely, the problem is that events can take a few seconds (or more) to be indexed. that is the reason that the normal practice is to schedule such a job to run a few minutes after the hour, rather than immediately on the hour.

Re: Why alert is not triggering on exact time

chandana204 — Tue, 14 Aug 2018 16:33:07 GMT

Thanks for your response. My alerts are triggering before the hour not after the hour. I'll use your query to make sure the selected time range.

Thanks,
Chandana