Finding the host for which the REST thread limit reached

[kteng2024]

Hi,

I could see the following warning :

" Can't handle request max thread limit for REST HTTP server"

Is there any way that we could know because of which hosts or queries we are getting this warning ?

[Masa]

First, I would check the thread limit.
http://docs.splunk.com/Documentation/Splunk/6.5.3/Troubleshooting/HTTPthreadlimitissues

Then, the number is high enough, and the issue persists, I would recommend to contact Splunk Support for further investigation.
Splunk Support will probably ask to collect diag, ps or top output with thread option, and pstack so that Splunk engineer will be able to look into thread stacks.

[kteng2024]

Thank you .. Is there any way that we could know which hosts are responsible for this warning ?

[Masa]

In splunkd.log, you can find a log. The host of the splunkd.log is using up threads limit.

Here is an example;
02-04-2016 12:11:08.983 -0800 WARN HttpListener - Can't handle request for $REST_request_here$, max thread limit for REST HTTP server is 1000, threads already in use is 1001

[s2_splunk]

Do you want to provide a bit more detail as to what exactly you are doing? Who is calling what via REST etc.
Without knowing a bit more about your setup, it'll be hard to help, I suspect.