This question is based on a comment from @woodcock on this post: https://community.splunk.com/t5/Splunk-Search/Why-are-real-time-searches-not-running-and-getting-err... in which the alert equation provided is as follows:
"Schedule it to cover a span of X and run it every X/2. This covers the case where events at the end of span t an the beginning of t+1 would just miss triggering in those windows but will hit in the next alert run. Then make X as large as you can stomach."
I do not fully understand this so I am hoping someone can help me out here.
Let's say I have an alert running every 5 mins. By that equation I should search -10m to now. But isn't that going to also significantly overlap with the prior run? Why not search -6m to now, for example?
How do span sizes affect things? Here is an alert I have running every 5 mins. I did notice the search itself picks up the current span and the prior span so I have been wondering how to optimize this properly.
| mstats avg(cpu_metric.pctIdle) as Idle WHERE index="itsi_im_metrics" AND host="*" span=5m by host
| eval cpu_utilization=round(100 - Idle,2)
| where cpu_utilization > 90
| stats list(host) as host_list list(cpu_utilization) as avg_cpu_utilization
