What causes delayed searches alerts in Splunk Enterprise - Error says "searches delayed"

[SamHTexas]

What do I need to check / do to resolve this please?

What causes delayed searches alerts in Splunk Enterprise - Error says "searches delayed"

[richgalloway]

Searches are delayed when there are no resources available at run-time and they have a non-zero Schedule Window.  The delay lasts until the schedule window closes.  If, at that time, the search still can't run then it becomes "skipped".

To resolve it, re-schedule the searches so fewer are scheduled at the same time.  Pay particular attention to the :00, :15, :30, and :45 minutes of each hour.  See https://github.com/dpaper-splunk/public/blob/master/dashboards/extended_search_reporting.xml for a helpful dashboard.

[SamHTexas]

Please tell me how to use the resource you listed o github. Thanks very much.

[isoutamo]

Just copy paste it to your node where you have those delayed searches as a dashboard.
Another option is use MC's Search -> Scheduler and look there what those searches are.
Anyhow you should look that time by time or create alert to inform you if there are lot of skipped or delayed searches.
r. Ismo

[SamHTexas]

Sir, what is the out come of using the github search you shared on a SH in Splunk. It ran for a while but no reports or messages appeared. Please advise. Thank you in advance.

[SamHTexas]

Sir, what is the out come of using the github search you shared on a SH in Splunk. It ran for a while but no reports or messages appeared. Please advise. Thank you in advance.

[SamHTexas]

Thank u for your message. I went to Monitoring console - Search - Scheduler Activity - Instance. All I see are "Search is waiting for input" in different windows. Please advise. Thx

[richgalloway]

Make sure each dropdown has something in it.  Verify the MC is running in distributed mode and that each search head is a search peer to the MC.

[isoutamo]

You could found more information about MC from https://docs.splunk.com/Documentation/Splunk/8.1.3/DMC/DMCoverview