Archive
Highlighted

Why the alert did not trigger for below cron expression?

Path Finder

16-59/10 5-6 * * * cron was setup for more than 0 events.

We had an event at 5:15 Am. Any idea why the alert did not trigger?

The query used is for -5m@m

Tags (1)
0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Ultra Champion

With that cron schedule, I guess the search ran first time at 5:20 AM? Did you confirm the search actually ran, and indeed ran at that time?

0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Path Finder

@FrankVI
Should not the search run at 5:16 and check for last 5 minutes? Also, how to check when the search ran at that time?

0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Ultra Champion

No, you set it to /10, so it runs at 0,10,20,30,40,50 (where 0 and 10 are skipped because of your 16-59 time window).

0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Path Finder

According to me,cron expression = 16-59/10 5-6 * * * means the search query will trigger at 5 hours and between 16 to 59 minutes in a span of 10 minutes, same for the hour 6.

So it will run,

5:16, 5:26, 5:36, 5:46, 5:56 and same for 6th hour

0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Path Finder

I just checked and confirmed that the it is scheduled 05:16:00

0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Ultra Champion

Hmm, I might be wrong about that then. I also checked with crontab guru and that agrees with you that it would run at 16,26,36,46,56 : https://crontab.guru/#16-59/10_5-6_*_*_*

Note: I added 2 stars at the end to make it a proper complete cron schedule.

From the settings page for saved searches, you should see a "View Recent" link in the actions column. Which allows you to inspect recent search executions. Also saved search executions are logged in index=_audit.

0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Esteemed Legend

Just because your event happened at that time does not mean that it was indexed and searchable at the time the search ran. A window so short as "within the last 5 minutes" leaves very little time for pipeline latencies which are common forwarding events into Splunk. If you compare the value of _time with _indextime for that event and they are more than 5-minutes apart (300 seconds), then the latency indicates that the event was not searchable in Splunk when the search looking for it ran.

0 Karma
Highlighted

Re: Why the alert did not trigger for below cron expression?

Esteemed Legend

And before @mattymo says, it: Meta W00t!

0 Karma