Deployment Architecture

How to add a new Search Head Cluster member to an existing SHC group if we have lost the secret key?

rbal_splunk
Splunk Employee
Splunk Employee

I have a 7 Node Search Head Cluster pool that was set up using a secret key. Unfortunately we lost secret and now we have a need to add another Member. What are my options?

0 Karma
1 Solution

rbal_splunk
Splunk Employee
Splunk Employee

Here are two options that worked with me…

Option 1 : Use Splunk to Decrypt Password Encrypted by Splunk : Refer :- https://hurricanelabs.com/blog/decrypt-passwords-encrypted-by-splunk/
Option 2 : in this approach follow
+Install the new instance of Search Head Cluster Member . (don’t start Splunk instance yet)
+Copy $SPLUNK_HOME_EXISTING_SHC/etc/auth/splunk.secret from one of the existing Search Head cluster to New instances $SPLUNK_HOME_NEW_SHC/etc/auth
+After that start splunk for the first time copy $SPLUNK_HOME_EXISTING_SHC/etc/system/local/server.conf stanza [shclustering] to $SPLUNK_HOME_NEW_SHC/etc/system/local/server.conf

[shclustering]
conf_deploy_fetch_url = https://DEPLOYER_URI:DEPLOYER_MGMT_PORT
disabled = 0
mgmt_uri = https://:SPLUNK_HOME_NEW_SHC_MGMT_PORT
pass4SymmKey = $1$XccY3P4=
replication_factor = 1
id = B1D0A95D-4DB6-4111-A702-57EADDFFC932

+Restart the Splunk new Search Head cluster Member.
+Follow the link to add it as Search Head Cluster member to exiting SHC group.

View solution in original post

rbal_splunk
Splunk Employee
Splunk Employee

Here are two options that worked with me…

Option 1 : Use Splunk to Decrypt Password Encrypted by Splunk : Refer :- https://hurricanelabs.com/blog/decrypt-passwords-encrypted-by-splunk/
Option 2 : in this approach follow
+Install the new instance of Search Head Cluster Member . (don’t start Splunk instance yet)
+Copy $SPLUNK_HOME_EXISTING_SHC/etc/auth/splunk.secret from one of the existing Search Head cluster to New instances $SPLUNK_HOME_NEW_SHC/etc/auth
+After that start splunk for the first time copy $SPLUNK_HOME_EXISTING_SHC/etc/system/local/server.conf stanza [shclustering] to $SPLUNK_HOME_NEW_SHC/etc/system/local/server.conf

[shclustering]
conf_deploy_fetch_url = https://DEPLOYER_URI:DEPLOYER_MGMT_PORT
disabled = 0
mgmt_uri = https://:SPLUNK_HOME_NEW_SHC_MGMT_PORT
pass4SymmKey = $1$XccY3P4=
replication_factor = 1
id = B1D0A95D-4DB6-4111-A702-57EADDFFC932

+Restart the Splunk new Search Head cluster Member.
+Follow the link to add it as Search Head Cluster member to exiting SHC group.

graju89
Path Finder

@rbal_splunk : Hi rbal, I have the same problem. But I dont have SCH. I am trying to add new search head into clustering. We have lost the secret key of the master. I tried your second options. But no luck. Can you help me on this?

Thanks,

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...