topic Re: Finding the host for which the REST thread limit reached in Splunk Search

Finding the host for which the REST thread limit reached

kteng2024 — Wed, 05 Apr 2017 17:58:32 GMT

Hi,

I could see the following warning :

" Can't handle request max thread limit for REST HTTP server"

Is there any way that we could know because of which hosts or queries we are getting this warning ?

Re: Finding the host for which the REST thread limit reached

s2_splunk — Wed, 05 Apr 2017 18:12:44 GMT

Do you want to provide a bit more detail as to what exactly you are doing? Who is calling what via REST etc.
Without knowing a bit more about your setup, it'll be hard to help, I suspect.

Re: Finding the host for which the REST thread limit reached

Masa — Wed, 05 Apr 2017 23:20:12 GMT

First, I would check the thread limit.
http://docs.splunk.com/Documentation/Splunk/6.5.3/Troubleshooting/HTTPthreadlimitissues

Then, the number is high enough, and the issue persists, I would recommend to contact Splunk Support for further investigation.
Splunk Support will probably ask to collect diag, ps or top output with thread option, and pstack so that Splunk engineer will be able to look into thread stacks.

Re: Finding the host for which the REST thread limit reached

kteng2024 — Thu, 06 Apr 2017 17:13:12 GMT

Thank you .. Is there any way that we could know which hosts are responsible for this warning ?

Re: Finding the host for which the REST thread limit reached

Masa — Tue, 29 Sep 2020 13:36:52 GMT

In splunkd.log, you can find a log. The host of the splunkd.log is using up threads limit.

Here is an example;
02-04-2016 12:11:08.983 -0800 WARN HttpListener - Can't handle request for $REST_request_here$, max thread limit for REST HTTP server is 1000, threads already in use is 1001