topic Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index in Getting Data In

-0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

rhirasin — Tue, 29 Sep 2020 14:31:27 GMT

$ tail -f splunkd.log
06-19-2017 06:08:12.823 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer
06-19-2017 06:08:16.540 -0500 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_10.207.140.100_8089_apsrs3355.uhc.com_apsrs3355_128161B5-DBCF-49EE-91B7-406544EB0CDA
06-19-2017 06:08:42.692 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer
06-19-2017 06:09:12.560 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer
06-19-2017 06:09:16.562 -0500 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_10.207.140.100_8089_apsrs3355.uhc.com_apsrs3355_128161B5-DBCF-49EE-91B7-406544EB0CDA
06-19-2017 06:09:42.437 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer
06-19-2017 06:09:49.310 -0500 WARN TcpOutputProc - Forwarding to indexer group indexers blocked for 3500 seconds.
06-19-2017 06:10:12.308 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer
06-19-2017 06:10:16.583 -0500 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_10.207.140.100_8089_apsrs3355.uhc.com_apsrs3355_128161B5-DBCF-49EE-91B7-406544EB0CDA
06-19-2017 06:10:42.177 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer
06-19-2017 06:11:12.050 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer
06-19-2017 06:11:16.606 -0500 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_10.207.140.100_8089_apsrs3355.uhc.com_apsrs3355_128161B5-DBCF-49EE-91B7-406544EB0CDA
06-19-2017 06:11:29.326 -0500 WARN TcpOutputProc - Forwarding to indexer group indexers blocked for 3600 seconds.
06-19-2017 06:11:41.924 -0500 ERROR TcpOutputFd - Read error. Connection reset by peer

Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

jkat54 — Mon, 19 Jun 2017 11:45:28 GMT

It looks like you're trying to send data to the indexer on port 8089. data input is usually on port 9997. Please make sure your specifying the correct forwarding port in your forwarder's outputs.conf.

Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

rhirasin — Mon, 19 Jun 2017 11:58:15 GMT

here is my output file. its look good. but still its not working

cat outputs.conf

[tcpout]

whitelist all event forwarding

forwardedindex.0.whitelist = .*

null out the following splunk defaults

forwardedindex.1.blacklist =
forwardedindex.2.whitelist =

defaultGroup = indexers

[tcpout:indexers]

Add all indexers on the following line. Separate indexers with a comma, if more than one exists.

server = server1.uhc.com:9997,server2.uhc.com:9997

Test Test

server = apsrd7043:9997

Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

jkat54 — Mon, 19 Jun 2017 14:51:01 GMT

run this command on the forwarder and post the details please.

./splunk btool outputs list --debug

Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

jkat54 — Mon, 19 Jun 2017 14:52:50 GMT

This is the indexer giving the error in the log:

apsrs3355

Yet you have apsrd7043 in the outputs you posted. So the btool command will show us which outputs.conf is pointing to apsrs3355, and then we can assist you on how to solve it... might be as simple as removing the other outputs.conf that points to apsrs3355.

Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

rhirasin — Mon, 19 Jun 2017 14:53:25 GMT

my issue go resolved. I have to use certs file in my secure env.

thanks for your quick help

Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

jkat54 — Mon, 19 Jun 2017 20:20:49 GMT

I converted your comment to answer. Please mark it as the answer.

Re: -0500 ERROR TcpOutputFd - Read error. Connection reset by peer. not not able to forward data into index

arunkchow — Thu, 14 Sep 2017 18:07:37 GMT

Our org has a small splunk setup. I am trying to secure the splunk with letsencrypt. I have the certs already and put them in /opt/splunk/etc/auth/certs path.
Lets encrypt issues the files as cert.pem, chain.pem, fullchain.pem and privkey.pem.
I pointed to the location of certs in both web.conf and server.conf under /opt/splunk/etc/system/local/ on indexer server and outputs.conf on forwarders.
But I am still getting the same error and forwarders don't forward any data.