How to connect Search Head Cluster members to Indexers Cluster Master?

[siemteam]

After check some different questions about this I cannot find a solution.

I'm trying to point my search head cluster to my indexers cluster using Cluster Master but when I use the following command on a SH Cluster member:

/opt/splunk/bin/splunk edit cluster-config -mode searchhead -master_uri https://cmserver:8089 -secret myKey

I recieve the following error message:

Could not contact master. Check that the master is up, the master_uri=https://cmserver:8089 and secret are specified correctly

splunkd.log on cluster master:

01-21-2019 18:25:15.530 +0100 ERROR DigestProcessor - Failed signature match
01-21-2019 18:25:15.531 +0100 ERROR LMHttpUtil - Failed to verify HMAC signature, uri: /services/cluster/master/info

I think that pass4SymmKey is equal on both servers, but how can I check it?

Thanks

[Marco-IT]

Hi @siemteam , did you solve the issue related to following messages?

01-21-2019 18:25:15.530 +0100 ERROR DigestProcessor - Failed signature match
01-21-2019 18:25:15.531 +0100 ERROR LMHttpUtil - Failed to verify HMAC signature, uri: /services/cluster/master/info

[dkeck]

HI,

you can decrypt passwords like that :

https://www.hurricanelabs.com/splunk-tutorials/make-splunk-do-it-how-to-decrypt-passwords-encrypted-...

sounds like they have different secrets files

[siemteam]

I'm not able to connect any search head cluster member to cluster master, instead of decrypt the passowrd (I haven't got other server to follow the process) I'm typing the password again in plain text at each server on /local/server.conf under [clustering] stanza but continue happening the same.

Any idea?

Thanks again

[dkeck]

Did you try it via UI?

[siemteam]

I cannot find steps to do it using UI, could you provide me a link please?

Thanks

[dkeck]

There you go

https://docs.splunk.com/Documentation/Splunk/6.4.1/Indexer/Enablethesearchhead#Enable_the_search_hea...

[siemteam]

At the end you can read the following:

If you want to deploy a search head cluster, so that the search heads share configurations and jobs, see the additional configuration instructions in the topic "Integrate the search head cluster with an indexer cluster" in the Distributed Search manual.

Is the case that I have, I'm trying to connect the members of the cluster.

[dkeck]

ah ok, I did not get that,

yes then you need https://docs.splunk.com/Documentation/Splunk/6.4.1/DistSearch/SHCandindexercluster

[siemteam]

Yes, Is the procedure that I'm following but at the moment of link to cluster master is not possible, I'm modifying the pass4SymmKey on each server but it's not working

[siemteam]

Thanks for the answer, I'm going to check the way to decrypt password.

When you talk about secrets file I undestand that the file /opt/splunk/etc/auth/splunk.secret, right?

What does this file exactly contains? is generated taking pass4SimmKey or it's different?

On the same folder I can find splunk.secret.oldKey and splunk.secret.oldKey.orig (maybe I generated these files indirectly trying to solve the issue)

Thank you so much

[dkeck]

yes, I am talking about the splunk.secret. Its used to hash the passwords you enter for pass4Symmkey.

I am not sure but this splunk.secret.oldKey does not sound like something that happens done by splunk. Maybe someone just changed this file on your server?