WHAT is this error "Search peer "sql01" has the following message: pass4SymmKey setting in the clustering or general stanza of server.conf is set to empty or the default value. You must change it to a different value."?

[akhil4mdev]

Detail :

The search head i am working is called "BLADE"

A customer stack has :

2 IDX (sql01,sql02)in a cluster with a cluster master.(sqlMaster1)
A search head(SH1,SH2) cluster with a deployer .

SO FOR EASY ACCESS :
we again attached the 2 IDX (sql01,sql02) of customer stack as search peers to "BLADE" and also added the customer stacks cluster master(sqlMaster1) under "indexer clustering" in "BLADE".

so sql01 went down
under "BLADE" -->distributed search --> search peers

so i deleted it and added it

now i can see it is up and running

But i see this error on searchhead "BLADE"

Search peer "sql01" has the following message: pass4SymmKey setting in the clustering or general stanza of server.conf is set to empty or the default value. You must change it to a different value.

[darrenfuller]

Hi @akhil4mdev,

On BLADE, run the following:

/opt/splunk/bin/splunk btool server list --debug | egrep "(\[(clustering|shclustering)\]|pass4SymmKey\s+\=)"

This will return something like the following:

/opt/splunk/etc/apps/org_cluster_search_base/default/server.conf       [clustering]
/opt/splunk/etc/system/default/server.conf                             pass4SymmKey =
/opt/splunk/etc/apps/org_cluster_search_base/local/server.conf         pass4SymmKey = $1$t7/fkqNtX2XLT7JoAA==
/opt/splunk/etc/system/default/server.conf                             pass4SymmKey =
/opt/splunk/etc/system/default/server.conf                             [shclustering]
/opt/splunk/etc/system/default/server.conf                             pass4SymmKey = 

Now, that error message is saying that one of either [clustering] or [shclustering] has not set a custom value. if you look at the example, i have a custom pass4SymmKey set on /opt/splunk/etc/apps/org_cluster_search_base/local/server.conf to set the value for [clustering], but for [shclustering], there is nothing other than the /etc/system/default value.... This one is the one that needs to be addressed.

The value for pass4SymmKey needs to match on all the hosts in a cluster as well as the deployer / cluster master.