topic Re: Unable to add search peer from search head using distributed search :no route to host or connection refused in Deployment Architecture

Unable to add search peer from search head using distributed search :no route to host or connection refused

RK_sp1unk — Thu, 06 Feb 2020 09:21:41 GMT

Issue:Unable to add search peer from search head using distributed search :no route to host or connection refused error

we have 5 instance

search head license master
indexer
search head enterprise security
heavy forwader
deployment server

all vm instances are created , we are now adding search peers from search head license master and search head enterprise security, the search peer would be indexer.

here from search head LM we cannot do a telnet to indexer using 8089 port, but vice versa it is working.

also telnet from search head Enterprise security to search head LM is also connecting

but we are unable to do telnet to indexer on port 8089 from both SH LM and SH ES.

while trying to add new peer if we put htttps://ipaddress:8089 we get error no route to host

if we put https://hostnameofindexer:8089 and add peer we get error connection refused

splunk version:8.0
vmware esxi
os:centos 8

This issue is very critical as whole project is stuck now.

Re: Unable to add search peer from search head using distributed search :no route to host or connection refused

nickhills — Thu, 06 Feb 2020 15:52:09 GMT

Everything in your post suggests that this is either a networking issue, or for some reason Splunk is not accepting connections.
The difference between the results of your browsers tests is just an artifact of how your browser reports failures for IP vs DNS name.

On your indexers, run netstat to confirm that the ports are open on 8089.
Confirm your SH can ping/route to indexers
Confirm you have no harware/application firewalls keeping connections out. On centos 8, you can try disabling the Firewall temporarily systemctl stop firewalld to see if that resolves the issue - remember to restart it and add rules if it does!

Re: Unable to add search peer from search head using distributed search :no route to host or connection refused

RK_sp1unk — Sun, 09 Feb 2020 06:21:59 GMT

on my indexer if do netstat , it shows the port is used by SHLM i.e
it shows TCP connection established
SIX.localdomain:47206 10.200.5.51:8089

i am getting the below errror while trying to add serch peer on SHLM andSHES

from SHLM i can ping and do telnet to indexer

from SHES also i can ping and do telnet to indexer

for remote user name and password , i am entering the admin username and password which i use to login to indexer web and which i created during the splunk installation , is this correct

error:Encountered the following error while trying to save: Peer with server name localhost.localdomain conflicts with this server's name.

disabling the firewall on indexer or Search heads ...please clarify...

tried disabling it on indexer no go...

please check this at priority as i am stuck now

Re: Unable to add search peer from search head using distributed search :no route to host or connection refused

RK_sp1unk — Sun, 09 Feb 2020 18:09:03 GMT

this issue is resolved it was a host name conflict