I have encountered errors from a host that is sending data to the splunk server(running on linux) via port 9997.All along it is working fine till last two days.
The error encountered in splunkd was:
Error TcpInputProc: Error encountered for connection from host=184.108.40.206,ip=220.127.116.11. Timeout.
So far there are no issues with other hosts on port 9997 except for this.
Does this indicate that the problem lies with the splunk server itself or the host that is sending data?
Am able to telnet from the host to the splunk server on port 9997.In this case there is no firewall blocking,which I initially suspect due to the iptables that i've enabled yesterday,but this host has stopped sending data since the last two days.
Run tcpdump or the like to sniff the traffic and correlate tcp events (and gaps in timestamps) with this error to see what your issue is. It could be an unrelated network issue that slows things down, but looking at the raw data and packet metadata may help.