Deployment Architecture

How to add a new Search Head Cluster member to an existing SHC group if we have lost the secret key?

rbal_splunk
Splunk Employee
Splunk Employee

I have a 7 Node Search Head Cluster pool that was set up using a secret key. Unfortunately we lost secret and now we have a need to add another Member. What are my options?

0 Karma
1 Solution

rbal_splunk
Splunk Employee
Splunk Employee

Here are two options that worked with me…

Option 1 : Use Splunk to Decrypt Password Encrypted by Splunk : Refer :- https://hurricanelabs.com/blog/decrypt-passwords-encrypted-by-splunk/
Option 2 : in this approach follow
+Install the new instance of Search Head Cluster Member . (don’t start Splunk instance yet)
+Copy $SPLUNK_HOME_EXISTING_SHC/etc/auth/splunk.secret from one of the existing Search Head cluster to New instances $SPLUNK_HOME_NEW_SHC/etc/auth
+After that start splunk for the first time copy $SPLUNK_HOME_EXISTING_SHC/etc/system/local/server.conf stanza [shclustering] to $SPLUNK_HOME_NEW_SHC/etc/system/local/server.conf

[shclustering]
conf_deploy_fetch_url = https://DEPLOYER_URI:DEPLOYER_MGMT_PORT
disabled = 0
mgmt_uri = https://:SPLUNK_HOME_NEW_SHC_MGMT_PORT
pass4SymmKey = $1$XccY3P4=
replication_factor = 1
id = B1D0A95D-4DB6-4111-A702-57EADDFFC932

+Restart the Splunk new Search Head cluster Member.
+Follow the link to add it as Search Head Cluster member to exiting SHC group.

View solution in original post

rbal_splunk
Splunk Employee
Splunk Employee

Here are two options that worked with me…

Option 1 : Use Splunk to Decrypt Password Encrypted by Splunk : Refer :- https://hurricanelabs.com/blog/decrypt-passwords-encrypted-by-splunk/
Option 2 : in this approach follow
+Install the new instance of Search Head Cluster Member . (don’t start Splunk instance yet)
+Copy $SPLUNK_HOME_EXISTING_SHC/etc/auth/splunk.secret from one of the existing Search Head cluster to New instances $SPLUNK_HOME_NEW_SHC/etc/auth
+After that start splunk for the first time copy $SPLUNK_HOME_EXISTING_SHC/etc/system/local/server.conf stanza [shclustering] to $SPLUNK_HOME_NEW_SHC/etc/system/local/server.conf

[shclustering]
conf_deploy_fetch_url = https://DEPLOYER_URI:DEPLOYER_MGMT_PORT
disabled = 0
mgmt_uri = https://:SPLUNK_HOME_NEW_SHC_MGMT_PORT
pass4SymmKey = $1$XccY3P4=
replication_factor = 1
id = B1D0A95D-4DB6-4111-A702-57EADDFFC932

+Restart the Splunk new Search Head cluster Member.
+Follow the link to add it as Search Head Cluster member to exiting SHC group.

graju89
Path Finder

@rbal_splunk : Hi rbal, I have the same problem. But I dont have SCH. I am trying to add new search head into clustering. We have lost the secret key of the master. I tried your second options. But no luck. Can you help me on this?

Thanks,

0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...