Getting Data In

Connection errors to heavy forwarders

ebaileytu
Communicator

we have the following setup

2 heavy forwarders (HF) forwarding data to 4 indexers

We just added another 100 Universal forwarders (UF) to the environment so now we have about 800 UFs connecting to the HFs. I am starting to see a troubling number of connection error messages (about 7000 per hour) from the UFs such as:

05-20-2014 21:10:16.949 -0500 ERROR TcpOutputFd - Connection to host=xx.xxx.xx.xx:xxxx failed. sock_error = 10054. SSL Error = error:00000000:lib(0):func(0):reason(0)

(We are using SSL for connections from the UF to HF)

and

05-20-2014 21:09:59.394 -0500 ERROR TcpOutputFd - Connection to host=xx.xxx.xx.xx:xxxx failed

Data is getting forwarded from the UF to the HF but from tests I can see some data is delayed. Do the errors indicate I need to adjust a setting or just deploy another HF? I do not see high resource utilization on the HF.

Thanks!

Tags (2)
0 Karma
1 Solution

ebaileytu
Communicator

issue was with the ESX server hosting the HF - very high iowait was the issue

View solution in original post

ebaileytu
Communicator

issue was with the ESX server hosting the HF - very high iowait was the issue

gsopko
New Member

Hi, what was the solution? 🙂

Thanks

0 Karma

ebaileytu
Communicator

issue with ESX server storage - high iowait created chaos

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...