topic Re: Maximum Number of Concurrent Running Jobs Search Head Cluster Error in Deployment Architecture

Maximum Number of Concurrent Running Jobs Search Head Cluster Error

ykpramodhcbt — Tue, 29 Sep 2020 17:12:44 GMT

Hi,

We have a search head cluster with 3 search heads -

We are seeing the following message and a decent skip ratio on DMC.

priority=default, status=skipped, reason="The maximum number of concurrent running jobs for this historical scheduled search on this cluster has been reached", concurrency_category="historical_scheduled", concurrency_context="saved-search_cluster-wide", concurrency_limit=1

This might be the way it is designed to run searches - only in one search head at a given time. If it turns out to be a bug, please mention how we can fix this.

regards
Pramodh

Re: Maximum Number of Concurrent Running Jobs Search Head Cluster Error

kamlesh_vaghela — Tue, 12 Dec 2017 08:10:18 GMT

HI @ykpramodhcbt,

Can you please check this answer?

https://answers.splunk.com/answers/337992/maxsearches-limit-reached.html

Re: Maximum Number of Concurrent Running Jobs Search Head Cluster Error

baldwintm — Thu, 14 Dec 2017 00:30:42 GMT

Do these events have a "savedsearch_name" in them? how often is that search scheduled to run?
I think that's the message that indicates that a saved search is trying to run, but a previous instance of that search is still running (splunk will only allow one instance of a savedsearch to run at a time).

for example: if a saved search is scheduled to run every minute, but it takes five minutes to run, then it will skip the next four times that it tries to run.

Re: Maximum Number of Concurrent Running Jobs Search Head Cluster Error

ykpramodhcbt — Fri, 22 Dec 2017 20:56:24 GMT

We observed this behavior when searches are skipped.

Re: Maximum Number of Concurrent Running Jobs Search Head Cluster Error

gayappanjan — Fri, 13 Sep 2019 07:15:57 GMT

Hi ,
Thankyou for the info
could you please provide solution for this type of issues?