topic Re: Why are there errors when configuring search head clustering with a deployer? in Deployment Architecture

Why are there errors when configuring search head clustering with a deployer?

Prakash493 — Sat, 20 Apr 2019 17:01:42 GMT

Hi , I am configuring a search head clustering with a deployer but ran in many issues:
as per the Splunk docs these are the steps I did :

First I set the search head deployer:
I have added a pass4symmkey on deployer under shcluster stanza

[general]
serverName = sh2
pass4SymmKey = $1$ir3GnxLQSyQCIHmqN+mx
[shclustering]
pass4SymmKey = $1$1WrJE7i8jQ+THZ0MWxYh

I kept this same key across all search heads on server.conf ([shclustering]

pass4SymmKey = $1$1WrJE7i8jQ+THZ0MWxYh)

Setting up search head cluster:
I ran this command in all search heads (total 3):

./splunk init shcluster-config -auth admin:password -mgmt_uri "https://vvvvv:8089" -replication_port 8079 -replication_factor 3 -conf_deploy_fetch_url https:/// (deployer_ip_address):8089 -secret  -shcluster_label

I run this command in all search heads (Total 3)
then I did restart.

Choosing captain:
after completing all the above steps I choose any one search head to make a captain;

Then I run this command :

./splunk bootstrap shcluster-captain -servers_list "URI:8089". -autgh admin:password

But after running its saying ,

error=This node seems to have already joined another cluster with below members: 'https://xxxx:8089'. First remove the member from the old cluster. Then run 'splunk clean
 raft' on the member to reuse it in a new cluster; server=https://xxxxx:8089, error=This node seems to have already joined another cluster with below members: 'https://xxxxx:8089'. First remove the me
mber from the old cluster. Then run 'splunk clean raft' on the member to reuse it in a new cluster.

I am not clear why I am getting this error first time I am setting on these new servers , when I am running to make a captain getting this error. Please help me

2) And what the difference between general pass4symmkey and shcluster pass4symmkey , my pass4symmkey the shclustering one is same across all cluster members not the pass4symmkey which is under general stanza , which one should need to same across all search heads ?

Thanks

Re: Why are there errors when configuring search head clustering with a deployer?

lakshman239 — Tue, 23 Apr 2019 08:19:19 GMT

Not sure if your issue is resolved. I assume you are following steps as per https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/SHCdeploymentoverview

pls note that the security_key (pass4SymmKey) under shclustering stanza is used and should be same both in your deployer and each member of the SHC. The key in general is auto-generated as part of splunk install.
I suggest, clean and re-initialize all the instances and follow the steps again and use splunk show shcluster-status to check the status of the cluster after boot-strap.

Re: Why are there errors when configuring search head clustering with a deployer?

VatsalJagani — Tue, 23 Apr 2019 13:57:01 GMT

@Prakash493,
Few points that you need to notice.

You should not update passSymmKey under [general]
You should not add encrypted passSymmKey under [shcluster], instead you need to add passKey in normal characters and do splunk restart, splunk will encrypt that.
Your searchhead cluster need also to be connected with indexer cluster for that you need to add [clustering] same as you might have done on indexer.
Rest steps are okay about choosing captain and all.

Check if you did something wrong and redeploy the shcluster.