topic Re: failure during peer node setup in Deployment Architecture

failure during peer node setup

nivedita_viswan — Mon, 28 Sep 2020 17:41:45 GMT

I get the following error when I try setting up my Peer node:

Failed to register with cluster master reason: failed method=POST path=/services/cluster/master/peers master=https://sjcsplunkmstr01-prod.tlcinternal.com:8089 rv=0 actual_response_code=500 expected_response_code=201 status_line=HTTP/1.1 500 Internal Server Error [ event=addPeer status=retrying AddPeerRequest: { _id= active_bundle_id=8ABFFC208D2DCE1B6478276AA0188FC5 add_type=Initial-Add base_generation_id=0 forwarderdata_rcv_port=9997 forwarderdata_use_ssl=0 mgmt_port=8089 name=3A231B5A-EE5D-4F6F-A690-4198A2DE24CC register_forwarder_address= register_replication_address= register_search_address= replication_port=8080 replication_use_ssl=0 replications= server_name=sjcsplunk01-prod.tlcinternal.com status=Up } ].

On the master node, the following message appears:

Failed to add peer 'guid=3A231B5A-EE5D-4F6F-A690-4198A2DE24CC server name=sjcsplunk01-prod.tlcinternal.com ip=10.128.16.163:8089' to the master. Error=site not in available_sites

I have gone through a post which suggests checking the port status. Ports 8080 and 8089 are open.
I was able to add one node to the cluster, only the 2nd node fails. However, the failing node was acting as a stand-alone indexer before, whereas the added node was a new installation.

What could be the reason for this failure?

Re: failure during peer node setup

dxu_splunk — Thu, 25 Sep 2014 01:18:08 GMT

what version of splunk are u running?

 Error=site not in available_sites

claims that the peer does not have the right 'site' set up, if multisite is turned on, every peer must have a 'site' that is listed in the master's 'available_sites' setting (server.conf). if multisite is turned off, the peers should not have a 'site' attribute

Re: failure during peer node setup

nivedita_viswan — Thu, 25 Sep 2014 17:22:13 GMT

We are using the latest version - 6.1.3 of Splunk. Multisite is turned off and peers dont have a 'site' attribute in the peer server.conf files.

Re: failure during peer node setup

dxu_splunk — Thu, 25 Sep 2014 19:04:45 GMT

is the peer not the same version as the master?

Re: failure during peer node setup

jayannah — Tue, 27 Jan 2015 15:22:21 GMT

Yesterday I had the similar issue. The rootcause was the indexer had the 6.0.2 and Cluster Master with 6.1.3. After upgrading the indexers to 6.1.3 (same version as CM), the problem resolved.

Re: failure during peer node setup

nivedita_viswan — Tue, 27 Jan 2015 18:28:36 GMT

You are right. The splunk versions were different. Having the same versions solved this problem.

Re: failure during peer node setup

z5pate — Thu, 23 Jul 2015 17:27:53 GMT

I am having the same issue as the above, but both master and peer nodes are on the same versions (6.2.4, build 271043). Any ideas on what is causing it?

Re: failure during peer node setup

dxu_splunk — Thu, 23 Jul 2015 20:22:58 GMT

do you have multisite turned on, but did not set the site on the indexer? (or vice versa)

Re: failure during peer node setup

z5pate — Tue, 29 Sep 2020 07:32:05 GMT

Mulitsite is not turned on. Here are the server.conf for Master and Peer nodes:

MASTER NODE:

[sslConfig]
sslKeysfilePassword = $1$cGranKIu+bsX

[lmpool:auto_generated_pool_download-trial]
description = auto_generated_pool_download-trial
quota = MAX
slaves = *
stack_id = download-trial

[lmpool:auto_generated_pool_forwarder]
description = auto_generated_pool_forwarder
quota = MAX
slaves = *
stack_id = forwarder

[lmpool:auto_generated_pool_free]
description = auto_generated_pool_free
quota = MAX
slaves = *
stack_id = free

[general]
pass4SymmKey = $1$JyaOwOdosukX
serverName = splunkp01
site = default

[clustering]
access_logging_for_heartbeats = 1
max_peer_build_load = 5
mode = master
pass4SymmKey = $1$Nz6D2+5m7t8=
replication_factor = 2
search_factor = 1

[lmpool:auto_generated_pool_enterprise]
description = auto_generated_pool_enterprise
quota = MAX
slaves = *
stack_id = enterprise

[license]
active_group = Enterprise

PEER NODE:

[sslConfig]
sslKeysfilePassword = $1$nG7D0aK47qLW

[lmpool:auto_generated_pool_download-trial]
description = auto_generated_pool_download-trial
quota = MAX
slaves = *
stack_id = download-trial

[lmpool:auto_generated_pool_forwarder]
description = auto_generated_pool_forwarder
quota = MAX
slaves = *
stack_id = forwarder

[lmpool:auto_generated_pool_free]
description = auto_generated_pool_free
quota = MAX
slaves = *
stack_id = free

[general]
pass4SymmKey = $1$yyKXjef+pfDW
serverName = splunkp02
site = default

[clustering]
master_uri = https://xxx.xxx.xxx.50:8089
mode = slave
pass4SymmKey = $1$2zqalu7w+cY=

[lmpool:auto_generated_pool_enterprise]
description = auto_generated_pool_enterprise
quota = MAX
slaves = *
stack_id = enterprise

[license]
active_group = Enterprise