Splunk Forwarder not sending logs

[pdash]

I get the following error:

10-09-2013 00:28:22.177 -0600 WARN TcpOutputFd - Connect to X.X.X.X:9997 failed. No connection could be made because the target machine actively refused it.
10-09-2013 00:28:22.177 -0600 ERROR TcpOutputFd - Connection to host=X.X.X.X:9997 failed
10-09-2013 00:28:22.177 -0600 WARN TcpOutputProc - Applying quarantine to ip=X.X.X.X port=9997 _numberOfFailures=3

Also this warning was found in splunkd

PipelineComponent - MetricsManager:probeandreport() took longer than seems reasonable (11141 milliseconds) in callbackRunnerThread. Might indicate hardware or splunk limitations.

[ncsantucci]

see http://wiki.splunk.com/Community:CallbackRunnerThreadTooLong

[kristian_kolb]

What do the logs look like on the indexer. I'm not sure that it's a connectivity problem. "Actively refused" sounds like a RST was sent by the indexer.

Running out of file descriptors?

[gfuente]

Hello

Did you verified that the firewall ports are open? It seems to be a connectivity problem

Try to do a telnet from the fw to the indexer, from the command line:

telnet indexer 9997

Regards