Splunk Search

Most of searches are getting deferred

Rukmani_Splunk
Path Finder

Hi all,
We are using SoS app for monitoring our schedules. We are working on reducing the schedules which are being skipped . But what is case about the deferred searches ? Its keep on increasing ? how to reduce them.
thoughts pls

Tags (1)

yannK
Splunk Employee
Splunk Employee

A deferred search is a search that couldn't be executed right now, because of the system or role search concurrency limit. Therefore they are executed a few seconds later. This is an expected behavior.
By example is you have a dashboard with 10 searches, but a limit of 6 concurrent searches, some panels will load after the first ones completed.

If you look at the audit logs, you can find how long they were deferred before being executed.
If a search is deferred too long, it will finally be skipped: skipped searches

The root cause are usually caused by :
- too many searches : you have too many searches (or heavy dashboard)
- non optimized scheduled searches taking long to run and overlapping.
- hardware limit : the indexers and search-head have not enough cpu core to handle high search concurrency. (check limits.conf), on 6.0 the formula for historical search concurrency = 6+ 1* (number of cores)

(hint, disable the deployment monitor app if you have it)

Rukmani_Splunk
Path Finder

Thanks a lot

0 Karma

MuS
Legend

more an advice then an answer, check out this answer on search scheduling http://answers.splunk.com/answers/33717/scheduled-searches-for-summary-index-does-not-run-no-skipped...

Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...