- Splunk Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- Connectivity between depolyment client and indexer
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Connectivity between depolyment client and indexer
I am receiving the following message in the splunkd log on my UF (windows)
01-11-2013 09:47:35.129 -0500 ERROR TcpOutputFd - Connection to host=x.x.x.x:9997 failed
01-11-2013 09:47:35.129 -0500 WARN TcpOutputProc - Applying quarantine to idx=x.x.x.x:9997 numberOfFailures=2
01-11-2013 09:48:04.141 -0500 INFO TcpOutputProc - Removing quarantine from idx=x.x.x.x:9997
01-11-2013 09:48:05.072 -0500 WARN TcpOutputFd - Connect to x.x.x.x:9997 failed. No connection could be made because the target machine actively refused it.
I can ping between the indexer and UF and nothing seems amiss. I cannot test with telnet b/c it is disabled. Any suggestions for troubleshooting is appreciated! Thanks!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I solved the issue...
I added the following line to the /etc/system/local/inputs.conf file on the indexer
[splunktcp://9997]
Connection_host = none
I ran this by splunk support and they indicated that this is a good fix and that they are experiencing a bug in DNS reverse lookups and this is a good workaround.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This worked for me!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yup...this morning I added the following entry to the inputs.conf file on the indexer
[splunktcp://9997] connection_host = none
I now seem to be receiving data...does this make sense to you?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
...and are you seeing the same error messages in your UF logs still?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Update: I can now telnet to and from client/server over 9997 but still not receiving data.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ayn the issue is with all instances. I have re-enabled telnet on the network and I can telnet to and from client/server over port 23 but I cannot over port 9997. It seems that when I restart the splundd a few times I get a full update and then it all stops again so I know it can connect at some point but then it is refused...
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Ayn. I will see if I cant get something to test it out. Peculiar thing is that everything was working fing until 2 days ago and to my knowledge nothing has changed.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In order to properly troubleshoot the issue you need some kind of tool to check the connection to port 9997 on the indexer. This kind of problem is very often linked to firewall issues, or configuration issues on the indexer (port 9997 is not configured to receive data from other Splunk instances). Do you have other instances that are working properly and it's just an issue with this specific instance?
Routing logs with Splunk OTel Collector for Kubernetes
Welcome to the Splunk Community!
Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM
