Why are we getting error "TcpOutputProc - Cooked connection...timed out" from our universal forwarder on an OS X machine?

[cpreasbeck]

Hello,

I currently have the following error with a universal forwarder on an OS X machine:

-0700 WARN  TcpOutputProc - Cooked connection to ip=IP:PORT timed out

I'm not sure why it is having issues sending logs to Splunk. We have linux machines that seem to be working fine. Anyway, here is my inputs and outputs conf files:

Inputs.conf

[monitor:///var/log/]

Outputs.conf

[tcpout]
defaultGroup=defaultindexer

[tcpout:defaultindexer]
server = IP:PORT

The connection is established and we can even see traffic to the indexer, but it is not the logs. I have looked at many articles with similar issues, but it doesn't seem to pertain to my issue. Any idea as to why we are having this problem?

[changux]

Definitely, is a firewall/network/configuration issue, please double check firewall rules/certs/network segments.
You can check some of previous answered questions with the same error:

http://answers.splunk.com/answers/218422/tcpoutputproc-cooked-connection-to-iptimed-out.html
http://answers.splunk.com/answers/38206/cooked-connection-timed-out.html

And this about digital certs:

http://wiki.splunk.com/Community:Splunk2Splunk_SSL_DefaultCerts

[rsennett_splunk]

First step is to take it literally... firewall blocking, different subnet (the route doesn't exist) etc. If it's isolated to the one machine, it's likely going to be something along those lines. If you were having it pop up often and in different places, then it would be suggesting that your indexer was overloaded and that it was perhaps time to scale.

So check the heck out of the connection and come back here with assurances that it "should" be and "could" be getting through but isn't...