How do I properly configure schedule search/cron/alert times?

jackjack — Mon, 25 Oct 2021 21:01:32 GMT

This question is based on a comment from @woodcock on this post: https://community.splunk.com/t5/Splunk-Search/Why-are-real-time-searches-not-running-and-getting-error-quot/m-p/281407 in which the alert equation provided is as follows:

"Schedule it to cover a span of X and run it every X/2. This covers the case where events at the end of span t an the beginning of t+1 would just miss triggering in those windows but will hit in the next alert run. Then make X as large as you can stomach."

I do not fully understand this so I am hoping someone can help me out here.

Let's say I have an alert running every 5 mins. By that equation I should search -10m to now. But isn't that going to also significantly overlap with the prior run? Why not search -6m to now, for example?

How do span sizes affect things? Here is an alert I have running every 5 mins. I did notice the search itself picks up the current span and the prior span so I have been wondering how to optimize this properly.

| mstats avg(cpu_metric.pctIdle) as Idle WHERE index="itsi_im_metrics" AND host="*" span=5m by host | eval cpu_utilization=round(100 - Idle,2) | where cpu_utilization > 90 | stats list(host) as host_list list(cpu_utilization) as avg_cpu_utilization

Re: How do I properly configure schedule search/cron/alert times?

jackjack — Wed, 27 Oct 2021 16:26:49 GMT

Based on Alert Scheduling Best Practices it is recommended to use a time window that matches up with the cron window.

"Both the search time range and the alert schedule span one hour, so there are no event data overlaps or gaps."

topic Re: How do I properly configure schedule search/cron/alert times? in Splunk Search

How do I properly configure schedule search/cron/alert times?

Re: How do I properly configure schedule search/cron/alert times?