Reporting

What are the main reasons for scheduled search latency?

pradeepkumarg
Influencer

The difference between the dispatched_time and scheduled_time on the scheduler logs--What is this exactly telling? What are the reasons for this delay?

Could it be because the cluster captain is busy? Is there a limit on how many schedule searches can the captain delegate ?
Is it because search head members are exhausted?
Is it because the indexers are busy? Does the search head members even have visibility into the indexer performance and CPU availability?

Any insight into this is greatly appreciated. Thank you.

Regards,
Pradeep

0 Karma

lguinn2
Legend

Whether you have a search head cluster or independent search heads, the "scheduler" is responsible for determining when reports run, and if necessary, which reports to skip. Here is some key documentation on the scheduler
Configure the priority of scheduled reports

The biggest reasons that a scheduled search fails to run is that the search head has too many simultaneous jobs to run. The documentation has a good description of how that happens and what you can do in the scheduler to manage it.

In a distributed environment, a search needs resources on both the search head and the indexer tier in order to execute. As a rule of thumb, search performance can be improved by adding more indexers, since indexers do the actual searching. While adding indexers does not increase the number of simultaneous searches, it does increase the speed of the searches so that more searches execute within a given time.

HTH

Get Updates on the Splunk Community!

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...

Want a chance to win $500 to the Splunk shop? Take our IT Incident Management Survey!

  Top Trends & Best Practices in Incident ManagementSplunk is partnering up with Constellation Research to ...