Alerting

scheduling best practices question

rainhailrob
Path Finder

Yes, I read the http://docs.splunk.com/Documentation/Splunk/6.1.1/Alert/Definescheduledalerts document, but I'm still somewhat confused with regard to the following:

Coordinate the alert's search schedule with the search time range. This prevents situations where event data is accidentally evaluated twice by the search (because the search time range exceeds the search schedule, resulting in overlapping event data sets), or not evaluated at all (because the search time range is shorter than the search schedule).

Schedule your alerting searches with at least 60 seconds of delay. This practice is especially important in distributed search Splunk implementations where event data might not reach the indexer precisely at the moment when it is generated. A delay ensures that you are counting all of your events, not just the ones that were quickest to get indexed.

So is this basically telling me I should not do less than 6 minute searches (allowing for 60sec delay)? Essentially. I'd like to know if something occurred in the past 5minutes or would the following work?

earliest: -5m@m
latest: now
cron expression: */5 * * * *

Tags (2)
0 Karma

somesoni2
Revered Legend

It basically says that you should allow sufficient buffer time for events to be written to indexes and while doing the schedule, the search time range should accommodate that buffer time. E.g. You want to search to run every 5 min and process last 5 min data, use something like this

earliest=-7m@m
latest=-2m@m             [2 min buffer time]
cron : 2-59/5 * * * *    [run the search on 3rd minute of the hour, like 2,7,12,17,...
                         minutes. This will allow you to capture data for full 5 min 
                         periods like 0-5 will get captured by search running on minute 7
                         (-7m to -2m) and 6-10 will get captured by search on 12(-7m to -2m).
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...