Getting Data In
Highlighted

cooked connection timed out?

Path Finder

I see some of these time outs in the /var/log/splunk/splunk.log
Is this something I should be concerned about? Does the forwarder try a resend? Is this a potential data loss? or if there's a retry, does it handle the resend gracefully?

01-13-2012 01:52:56.098 +0000 INFO  TcpOutputProc - Connected to idx=x.x.x.x:9997
01-13-2012 01:54:15.760 +0000 WARN  TcpOutputProc - Cooked connection to ip=x.x.x.x:9997 timed out
01-13-2012 01:54:15.762 +0000 INFO  TcpOutputProc - Connected to idx=x.x.x.x:9997
01-13-2012 01:54:45.592 +0000 WARN  TcpOutputProc - Cooked connection to ip=x.x.x.x:9997 timed out
01-13-2012 01:55:15.423 +0000 WARN  TcpOutputProc - Cooked connection to ip=x.x.x.x:9997 timed out
01-13-2012 01:55:15.424 +0000 INFO  TcpOutputProc - Connected to idx=x.x.x.x:9997
01-13-2012 01:55:34.333 +0000 INFO  TcpOutputProc - Connected to idx=x.x.x.x:9997

Thanks!

Tags (3)
Highlighted

Re: cooked connection timed out?

Path Finder

having the same issue, data is not being forwarded. did you find what was the issue?

0 Karma
Highlighted

Re: cooked connection timed out?

Legend

If the issue is persistent, I suspect your forwarder is configured to setup an unencrypted connection to the indexer but the indexer only accepts encrypted connections - or vice versa.

Highlighted

Re: cooked connection timed out?

Path Finder

Thank you for your response, Ayn. I have both forwarder and receiver to use ssl... and I have done it many times... not sure what is different this time.

I have case opened, will see:
Description: WARN TcpOutputProc - Cooked connection to ip=xx.xx.xx.xx:9992 timed out

forwarder:
-bash-3.2# cat outputs.conf
[tcpout]
defaultGroup = splunkssl-LB

[tcpout:splunkssl-LB]
server = splunk06:9992
compressed = true

[tcpout-server://splunk06:9992]
sslCertPath = $SPLUNKHOME/etc/certs/forwarder.pem
sslCommonNameToCheck = indexer
sslPassword = xxxxxxxxxxxxxxxx
sslRootCAPath = $SPLUNK
HOME/etc/certs/cacert.pem
sslVerifyServerCert = true

receiver:
# HOST
[default]
host = splunk06

[SSL]
password = xxxxxxxxxxxxxxxxxxx
requireClientCert = true
rootCA = $SPLUNKHOME/etc/certs/cacert.pem
serverCert = $SPLUNK
HOME/etc/certs/indexer.pem

[splunktcp-ssl://9992]
compressed = true

0 Karma
Highlighted

Re: cooked connection timed out?

Path Finder

found the issue - missing indexer cert
I wish it was easier to find in the forwarder log why connection was timing out...

closing the case. thank you!

0 Karma
Highlighted

Re: cooked connection timed out?

Explorer

I do not understand how this solved the issue. I have the issue sometimes (as seems to be the case in your question), that is, not always. Simply a misisng certificate would mean the problem should always happen, right?

Highlighted

Re: cooked connection timed out?

Legend

Excellent! Please mark some answer here as accepted, it shows that the "case is closed" so to speak 🙂

0 Karma
Highlighted

Re: cooked connection timed out?

Explorer

It would be better to post the correct answer as a new answer, and then mark it...

0 Karma
Highlighted

Re: cooked connection timed out?

Contributor

I just setup my new implementation and I am getting this error message intermittently for one of the three indexers, which is randomly "picked" to report this error. Forwarders and Indexers are otherwise communicating properly. What else might I look for as possible cause for this situation?

0 Karma
Highlighted

Re: cooked connection timed out?

SplunkTrust
SplunkTrust

thanks Ayn, you saved my day once again 😉

0 Karma