Detail :
The search head i am working is called "BLADE"
A customer stack has :
2 IDX (sql01,sql02)in a cluster with a cluster master.(sqlMaster1)
A search head(SH1,SH2) cluster with a deployer .
SO FOR EASY ACCESS :
we again attached the 2 IDX (sql01,sql02) of customer stack as search peers to "BLADE" and also added the customer stacks cluster master(sqlMaster1) under "indexer clustering" in "BLADE".
so sql01 went down
under "BLADE" -->distributed search --> search peers
so i deleted it and added it
now i can see it is up and running
But i see this error on searchhead "BLADE"
Search peer "sql01" has the following message: pass4SymmKey setting in the clustering or general stanza of server.conf is set to empty or the default value. You must change it to a different value.
Hi @akhil4mdev,
On BLADE, run the following:
/opt/splunk/bin/splunk btool server list --debug | egrep "(\[(clustering|shclustering)\]|pass4SymmKey\s+\=)"
This will return something like the following:
/opt/splunk/etc/apps/org_cluster_search_base/default/server.conf [clustering]
/opt/splunk/etc/system/default/server.conf pass4SymmKey =
/opt/splunk/etc/apps/org_cluster_search_base/local/server.conf pass4SymmKey = $1$t7/fkqNtX2XLT7JoAA==
/opt/splunk/etc/system/default/server.conf pass4SymmKey =
/opt/splunk/etc/system/default/server.conf [shclustering]
/opt/splunk/etc/system/default/server.conf pass4SymmKey =
Now, that error message is saying that one of either [clustering] or [shclustering] has not set a custom value. if you look at the example, i have a custom pass4SymmKey set on /opt/splunk/etc/apps/org_cluster_search_base/local/server.conf to set the value for [clustering], but for [shclustering], there is nothing other than the /etc/system/default value.... This one is the one that needs to be addressed.
The value for pass4SymmKey needs to match on all the hosts in a cluster as well as the deployer / cluster master.