topic Re: Splunk Forwarder not sending logs in Getting Data In

Splunk Forwarder not sending logs

pdash — Thu, 10 Oct 2013 06:35:03 GMT

I get the following error:

10-09-2013 00:28:22.177 -0600 WARN TcpOutputFd - Connect to X.X.X.X:9997 failed. No connection could be made because the target machine actively refused it.
10-09-2013 00:28:22.177 -0600 ERROR TcpOutputFd - Connection to host=X.X.X.X:9997 failed
10-09-2013 00:28:22.177 -0600 WARN TcpOutputProc - Applying quarantine to ip=X.X.X.X port=9997 _numberOfFailures=3

Also this warning was found in splunkd

PipelineComponent - MetricsManager:probeandreport() took longer than seems reasonable (11141 milliseconds) in callbackRunnerThread. Might indicate hardware or splunk limitations.

Re: Splunk Forwarder not sending logs

gfuente — Thu, 10 Oct 2013 06:40:58 GMT

Hello

Did you verified that the firewall ports are open? It seems to be a connectivity problem

Try to do a telnet from the fw to the indexer, from the command line:

telnet indexer 9997

Regards

Re: Splunk Forwarder not sending logs

kristian_kolb — Thu, 10 Oct 2013 08:11:38 GMT

What do the logs look like on the indexer. I'm not sure that it's a connectivity problem. "Actively refused" sounds like a RST was sent by the indexer.

Running out of file descriptors?

Re: Splunk Forwarder not sending logs

ncsantucci — Wed, 06 May 2015 01:38:04 GMT

see http://wiki.splunk.com/Community:CallbackRunnerThreadTooLong