Getting Data In

Connection closes with INFO TcpOutputProc - Detected connection to 10.x.x.x:9997 closed.

mravindra
Engager

TCP connection closes after few hours and will not re-establish even after splunk restart.
Connection gets re-established by editing outputs.conf and again closes with below logs after few hours.

05-23-2018 00:29:36.512 +0200 ERROR TcpOutputFd - Connection to host=10.x.x.x:9997 failed. sock_error = 0. SSL Error = error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
05-23-2018 00:29:36.512 +0200 INFO TcpOutputProc - Detected connection to 10.x.x.x:9997 closed
05-23-2018 00:29:36.512 +0200 INFO TcpOutputProc - Will close stream to current indexer 10.x.x.x:9997
05-23-2018 00:29:36.512 +0200 INFO TcpOutputProc - Closing stream for idx=10.x.x.x:9997
05-23-2018 00:29:36.767 +0200 ERROR TcpOutputFd - Connection to host=10.x.x.x:9997 failed. sock_error = 0. SSL Error = error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
05-23-2018 00:30:06.567 +0200 ERROR TcpOutputFd - Connection to host=10.x.x.x:9997 failed. sock_error = 0. SSL Error = error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed

0 Karma

renjith_nair
Legend

Hi @mravindra,

If its between forwarder and indexer, please check the ssl configuration. looks like your server is not trusting the certificate presented by client. It could be because of various reasons including host name validation, common name mismatch etc.

http://docs.splunk.com/Documentation/Splunk/7.1.1/Security/ConfigureSplunkforwardingtousesignedcerti...

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

SasiB137
Engager

Right but the scenario here is "TCP connection closes after few hours and will not re-establish even after splunk restart." So not sure if it's a valid observation.

0 Karma

SasiB137
Engager

If we corrupt the forwarder certificate we should see the above logs getting emitted? Right ?

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...