Splunk Enterprise Security
Highlighted

Splunk Searches delayed

Builder

Hi All,
I would like to ask why do we encounter this notification:
Root Cause(s):

  • The percentage of high priority searches delayed (16%) over the last 24 hours is very high and exceeded the red thresholds (10%) on this Splunk instance. Total Searches that were part of this percentage=12. Total delayed Searches=2
  • The percentage of non high priority searches delayed (47%) over the last 24 hours is very high and exceeded the red thresholds (20%) on this Splunk instance. Total Searches that were part of this percentage=21. Total delayed Searches=10

May i know what are the possible issue and resolution regarding this?

0 Karma
Highlighted

Re: Splunk Searches delayed

Builder

I answered a similar question generally here - https://answers.splunk.com/answers/786499/the-percentage-of-non-high-priority-searches-lagge.html#an.... The gist is that you can use the Monitoring Console (and it's inherent queries) to better diagnose specifically what your issues are.

Here's the path (assuming you're a Splunk admin on your instance): Settings (Top right) -> Monitoring Console -> Search -> Scheduler Activity: Instance, and inputting the timeframe when this occurred. Hopefully the information under "historical charts" can point you in the direction of what caused this to occur (perhaps the machine blipped, you have a misconfigured search etc), or at least narrow down the timeframe/options so you can continue debugging.

Hope this helps!

View solution in original post

0 Karma
Highlighted

Re: Splunk Searches delayed

Builder

Hi Thanks for this, I manage to identify the issue.

Resolution:
Increase the Limits.conf base on server information and Splunk transactions.

0 Karma
Highlighted

Re: Splunk Searches delayed

Explorer

Hi jadengoho.
Can you please explain what configuration was added/extended in limits.conf to resolve this?

Thanks

0 Karma
Highlighted

Re: Splunk Searches delayed

Explorer

Even we have same issue.Can you please tell which attributes value should increase.?

 

 

0 Karma
Highlighted

Re: Splunk Searches delayed

Explorer

can you please tell me which attribute consider in limits.com.we have same issue.

0 Karma