Getting Data In

Connection errors to heavy forwarders

ebaileytu
Communicator

we have the following setup

2 heavy forwarders (HF) forwarding data to 4 indexers

We just added another 100 Universal forwarders (UF) to the environment so now we have about 800 UFs connecting to the HFs. I am starting to see a troubling number of connection error messages (about 7000 per hour) from the UFs such as:

05-20-2014 21:10:16.949 -0500 ERROR TcpOutputFd - Connection to host=xx.xxx.xx.xx:xxxx failed. sock_error = 10054. SSL Error = error:00000000:lib(0):func(0):reason(0)

(We are using SSL for connections from the UF to HF)

and

05-20-2014 21:09:59.394 -0500 ERROR TcpOutputFd - Connection to host=xx.xxx.xx.xx:xxxx failed

Data is getting forwarded from the UF to the HF but from tests I can see some data is delayed. Do the errors indicate I need to adjust a setting or just deploy another HF? I do not see high resource utilization on the HF.

Thanks!

Tags (2)
0 Karma
1 Solution

ebaileytu
Communicator

issue was with the ESX server hosting the HF - very high iowait was the issue

View solution in original post

ebaileytu
Communicator

issue was with the ESX server hosting the HF - very high iowait was the issue

gsopko
New Member

Hi, what was the solution? 🙂

Thanks

0 Karma

ebaileytu
Communicator

issue with ESX server storage - high iowait created chaos

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

REGISTER NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more ...

Security Highlights | November 2022 Newsletter

 November 2022 2022 Gartner Magic Quadrant for SIEM: Splunk Named a Leader for the 9th Year in a RowSplunk is ...

Platform Highlights | November 2022 Newsletter

 November 2022 Skill Up on Splunk with our New Builder Tech Talk SeriesCan you build it? Yes you can! *play ...