Connection Timed out and An existing connection was forcibly closed by the remote host

[vino06]

Hi Guys,

I am just a newbie in Splunk and this will be my first time to perform troubleshooting. I'm having a connection timed out with 6 of our servers and I think this is reason why there is no logs being forwarded to our Indexers. Also there is an error saying that "An existing connection was forcibly closed by the remote host". Hope anyone can help me on how to resolve this issue. Please see the screenshot below for reference.

[lfedak_splunk]

Hey @vino06, If cusello solved your problem, please don't forget to accept an answer! You can upvote posts as well. (Karma points will be awarded for either action.) Happy Splunking!

[gcusello · Accepted solution]

Hi vino06,
no there is another problem because Splunk Forwarders continuously send internal logs to Indexerds so the channel is used.

The problem is another: at first, are you sure about the available network bandwidth?

In addition: what storage do you used, in other words, disks are quick or not?
One usual problem of timeout is that Indexer is overloaded so cannot reach to index logs and put in wait transmission.
In these cases Forwarder caches its logs and send it as soon as connection is available, so you don't loose data.

Check performances and hardware requirements of your Indexer.

Bye.
Giuseppe

[vino06]

I already seek assistance with our FW team to check the connection of the servers going to our Indexers. Also i think your right saying that our Indexer is overload since we usually encounter this which results to "No Result found" on some of our server as well. But how can I fix the Connection Timed our or the Existing Connection has forcibly closed by remote host?

[gcusello]

Hi vino06,
Using Forwarders you don't loose events because it locally caches them and then sends them to Indexers as soon it's available.
Check if this is true in your situation, in other words see if you have in your indexer all the events of your files.
Anyway you can use Distributed Monitoring Console to check the Indexers health and its indexing load.
Bye.
Giuseppe