Getting Data In

Connection closes with INFO TcpOutputProc - Detected connection to 10.x.x.x:9997 closed.

mravindra
Engager

TCP connection closes after few hours and will not re-establish even after splunk restart.
Connection gets re-established by editing outputs.conf and again closes with below logs after few hours.

05-23-2018 00:29:36.512 +0200 ERROR TcpOutputFd - Connection to host=10.x.x.x:9997 failed. sock_error = 0. SSL Error = error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
05-23-2018 00:29:36.512 +0200 INFO TcpOutputProc - Detected connection to 10.x.x.x:9997 closed
05-23-2018 00:29:36.512 +0200 INFO TcpOutputProc - Will close stream to current indexer 10.x.x.x:9997
05-23-2018 00:29:36.512 +0200 INFO TcpOutputProc - Closing stream for idx=10.x.x.x:9997
05-23-2018 00:29:36.767 +0200 ERROR TcpOutputFd - Connection to host=10.x.x.x:9997 failed. sock_error = 0. SSL Error = error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
05-23-2018 00:30:06.567 +0200 ERROR TcpOutputFd - Connection to host=10.x.x.x:9997 failed. sock_error = 0. SSL Error = error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed

0 Karma

renjith_nair
Legend

Hi @mravindra,

If its between forwarder and indexer, please check the ssl configuration. looks like your server is not trusting the certificate presented by client. It could be because of various reasons including host name validation, common name mismatch etc.

http://docs.splunk.com/Documentation/Splunk/7.1.1/Security/ConfigureSplunkforwardingtousesignedcerti...

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

SasiB137
Engager

Right but the scenario here is "TCP connection closes after few hours and will not re-establish even after splunk restart." So not sure if it's a valid observation.

0 Karma

SasiB137
Engager

If we corrupt the forwarder certificate we should see the above logs getting emitted? Right ?

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...