topic Re: Searches Delayed_exceeded the yellow thresholds in Splunk Enterprise

Why is Searches Delayed_exceeded the yellow thresholds?

dhans2022 — Wed, 04 May 2022 16:36:13 GMT

The percentage of non high priority searches delayed (19%) over the last 24 hours is very high and exceeded the yellow thresholds (10%) on this Splunk instance. Total Searches that were part of this percentage=5927. Total delayed Searches=1141

Can anyone help me out.

Re: Searches Delayed_exceeded the yellow thresholds

Stefanie — Wed, 04 May 2022 13:53:51 GMT

Take a look at the Monitoring Console to see what searches are being delayed and why. Go to Monitoring Console -> Search -> Scheduler Activity: Instance and take a look at the "Count of Skipped Reports by Name and Reason"
It could be there are multiple searches running at the same time.

What type of server is having this error? Is it a normal Search Head or is it a Search Head running correlational searches (Ex: Enterprise Security)?

Re: Searches Delayed_exceeded the yellow thresholds

dhans2022 — Thu, 05 May 2022 04:43:48 GMT

Yes, facing this issue in Enterprise Security.

Re: Searches Delayed_exceeded the yellow thresholds

Stefanie — Tue, 10 May 2022 12:56:26 GMT

Since it is affecting Enterprise Security I would assume that you have many correlational searches enabled. 🙂
Are you familiar with Correlational searches?

It sounds like you may need to disable the searches that are not required for your Use case.