Reporting

Why do we have 75,000 skipped scheduled searches over the last 7 days using Splunk 6.0.1?

dcroteau
Splunk Employee
Splunk Employee

Below are numbers from the search heads which support our Alerting process through Splunk.

So over the last 7 days, We have missed 75,000 schedule searches. I don't understand this because we have less than 250 scheduled searches. We need help to resolve this issue. Please let me know what "best practice action" I should take and what to check to mitigate this skipped searches. All of these searches seem to be running as admin in the Search App.

Each searchhead has 4 core x 6 CPU’s
XXXXXXXm2p had 72,000 skipped searches over the last 7 days
XXXXXXXw2p had 3,600 skipped searches over the last 7 days.

What does Skipped really mean?

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Skipped typically means a scheduled search did not finish before its next scheduled run should start - then that next run is skipped to avoid loading up the queue infinitely.

In other words, your scheduler could not run all the searches it was scheduled to run. This can be caused by scheduled searches actually running for much longer than expected, or by other searches blocking too much resources, or by any other resource hog.

View solution in original post

Jarohnimo
Builder

Most of these searches i found are system generated. Perhaps from a Enterprise app that goes haywire with the scheduling. I'm not sure why this isn't better controlled by splunk as out of the box functions are being skipped.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Skipped typically means a scheduled search did not finish before its next scheduled run should start - then that next run is skipped to avoid loading up the queue infinitely.

In other words, your scheduler could not run all the searches it was scheduled to run. This can be caused by scheduled searches actually running for much longer than expected, or by other searches blocking too much resources, or by any other resource hog.

Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...