https://community.splunk.com/t5/Getting-Data-In/Why-am-I-getting-a-HTTP-503-error-when-using-threads-to-call/m-p/370252#M67190 <P>Hi All,</P> <P>I am using the Java splunk api service to make oneShotSearch calls for service data. <BR /> HTTP 503 response: <CODE>Search not executed: The maximum number of concurrent historical searches on this instance has been reached.", concurrency_category="historical", concurrency_context="instance-wide", current_concurrency=34, concurrency_limit=34</CODE></P> <P>What I am doing is creating multiple threads which perform the Service#oneshotSearch in given time intervals. It runs normally, but eventually, I get the HTTP 503 above error. I am not that familiar with Splunk (mainly focusing on the Java code) so could someone please elaborate on what this exactly means?</P> <P>I am confused what exactly instance is referring to, is this referring to a specific host included in the queries, or is this referring to my own user searches once I am logged in? I do not have 34 threads, so are they getting backed up and queued through the api?</P> <P>If anyone could please provide me with some details on this error to help troubleshoot, I would greatly appreciate it. Thank You!</P> Thu, 08 Feb 2018 18:15:29 GMT Aweave15 2018-02-08T18:15:29Z https://community.splunk.com/t5/Getting-Data-In/Why-am-I-getting-a-HTTP-503-error-when-using-threads-to-call/m-p/370252#M67190 <P>Hi All,</P> <P>I am using the Java splunk api service to make oneShotSearch calls for service data. <BR /> HTTP 503 response: <CODE>Search not executed: The maximum number of concurrent historical searches on this instance has been reached.", concurrency_category="historical", concurrency_context="instance-wide", current_concurrency=34, concurrency_limit=34</CODE></P> <P>What I am doing is creating multiple threads which perform the Service#oneshotSearch in given time intervals. It runs normally, but eventually, I get the HTTP 503 above error. I am not that familiar with Splunk (mainly focusing on the Java code) so could someone please elaborate on what this exactly means?</P> <P>I am confused what exactly instance is referring to, is this referring to a specific host included in the queries, or is this referring to my own user searches once I am logged in? I do not have 34 threads, so are they getting backed up and queued through the api?</P> <P>If anyone could please provide me with some details on this error to help troubleshoot, I would greatly appreciate it. Thank You!</P> Thu, 08 Feb 2018 18:15:29 GMT https://community.splunk.com/t5/Getting-Data-In/Why-am-I-getting-a-HTTP-503-error-when-using-threads-to-call/m-p/370252#M67190 Aweave15 2018-02-08T18:15:29Z https://community.splunk.com/t5/Getting-Data-In/Why-am-I-getting-a-HTTP-503-error-when-using-threads-to-call/m-p/370253#M67191 <P>Open the job inspector (top right menu) and check how many searches are running.<BR /> You may have scheduled searches running<BR /> there are 2 limits that may kick in :<BR /> - quota of number of concurrent searches per user (see the roles, and authorize.conf)<BR /> - quota of the system maximum number of concurrent searches (see number of cores and limits.conf)</P> <P>And those limits are different for historical searches and real time searches.<BR /> see <A href="http://wiki.splunk.com/Community:TroubleshootingSearchQuotas">http://wiki.splunk.com/Community:TroubleshootingSearchQuotas</A><BR /> have a look at <A href="https://answers.splunk.com/answers/607068/the-maximum-number-of-concurrent-historical-search.html">https://answers.splunk.com/answers/607068/the-maximum-number-of-concurrent-historical-search.html</A></P> Thu, 08 Feb 2018 18:45:00 GMT https://community.splunk.com/t5/Getting-Data-In/Why-am-I-getting-a-HTTP-503-error-when-using-threads-to-call/m-p/370253#M67191 493669 2018-02-08T18:45:00Z