Searchhead is unable to update the peer information. Error = 'Unable to reach master'

[sarvesh_11]

Hi Splunkers,
I am configuring a new standalone search head, while configuration i am getting below warning,did my work around but unable to rectify it.

The searchhead is unable to update the peer information. Error = 'Unable to reach master' for master=https://Masterserver.com:8089.

My Indexers are in Cluster and in Splunk Cloud, and i am using this Standalone search head, this is in AWS Cloud.

Though i am not sure, below log is helpful in context to this, but this is the log i am repeatedly getting in /opt/splunk/var/log/splunk/splunkd.log

ERROR ClusteringMgr - VerifyMultisiteConfig failed Error=failed method=GET path=/services/cluster/master/info/?output_mode=json master=MasterServer.com:8089 rv=0 gotConnectionError=0 gotUnexpectedStatusCode=1 actual_response_code=401 expected_response_code=2xx status_line="Unauthorized" socket_error="No error" remote_error=call not properly authenticated

Thanks in Advance.

[DavidHourani · Accepted solution]

Hi @sarvesh_11,

You're getting error code 401 meaning u have an authentication issue.

Try re-entering you're pass4symmkey and restarting your SH.

Cheers,
David

[sarvesh_11]

Hey @DavidHourani i did this is server.conf on SH but no change, error is still the same.

[DavidHourani]

Any cert configured ?

Try the solution shown here :
https://answers.splunk.com/answers/562198/error-clusteringmgr-verifymultisiteconfig-failed-w-1.html

Before starting splunk, try to backup server.conf , then remove the sslConfig stantza in the server.conf and start the splunk.

[sarvesh_11]

Yeah @DavidHourani tried this too, removed the sslconfig stanza from /local/server.conf.
Yet no luck

[DavidHourani]

what do you get when you run : nc -vv Your_CM_IP 8089 from your SH

[sarvesh_11]

@DavidHourani
Ncat: Version 7.50 ( https://nmap.org/ncat )
NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt.
NCAT DEBUG: Unable to load trusted CA certificates from /usr/share/ncat/ca-bundle.crt: error:02001002:system library:fopen:No such file or directory
libnsock nsi_new2(): nsi_new (IOD #1)
libnsock nsock_connect_tcp(): TCP connection requested to CM_IP:8089 (IOD #1) EID 8
libnsock nsock_trace_handler_callback(): Callback: CONNECT SUCCESS for EID 8 [CM_IP:8089]
Ncat: Connected to CM_IP:8089.

THis looks fine, than definitely issue with pass4symmkey

[DavidHourani]

seems like it, yes