I'm looking for information or suggestions about how to proceed with this.
When configuring a cluster in Splunk, it gives you the option of entering a "Secret Key" on each of the machines that'll be joined in as part of the cluster.
I have two scenarios I'd like some suggestions for, though I think the process might be the same for each:
What would the best way to approach this be, without disrupting the cluster and the indexing process much?
I would also be interested if there is a way to decrypt the current secret key. I am trying to add a new SH cluster that points to our singular indexer cluster but don't have the secret keys. Knowing it would be FAAR easier than changing it and it is needed to point the new SH cluster to the indexer cluster
Decrypting a password or Pass4SymmKey in a conf file is unlikely to ever be possible because it would unravel Splunk's security system. I believe the practice is change the Pass4SymmKey everywhere. It may be painful but it's less painful than the security implications of undoing the Pass4SymmKey.
The following documentation has subsections following this page that explain how to configure the secret key for a cluster via dashboard, server.conf and CLI.
This documentation covers making changes to a cluster configuration, including the secret key and the various ways of doing so as well.
Hope this helps!