Why can I not push config from deployer to search head cluster?

[erw550]

Hello,

I recently upgraded our deployer/deployment server from 8.1.6 to version 9.0 and when I try to push configuration to our search head cluster i get an error that I have not seen before:

[splunk@aa130XXXXX bin]$ ./splunk apply shcluster-bundle -target https://aa130XXXXX:8089

 Warning: Depending on the configuration changes being pushed, this command might initiate a rolling restart of the cluster members.  Please refer to the documentation for the details. Do you wish to continue? [y/n]: y

WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

Your session is invalid.  Please login.

Splunk username: XXXXX

Password: 

Error in pre-deploy check, uri=https://aa130XXXXX:8089/services/shcluster/captain/kvstore-upgrade/status, status=401, error=No error

Our search head cluster is still on version 8.1.6

Thanks!

[saurabh_ha]

Hello,

I faced the same issue, following are the troubleshooting steps has been followed.

As ERROR=401 that unauthrized request is made from deployer to splunk SHC

Go to the /splunk-home/etc/system/local/server.conf of all SH cluster member and deployer and change the following, keep the value the same.

[shclustering]

pass4SymmKey=<clear_text_string>

Restart splunkd.

/opt/splunk/bin/splunk apply shcluster-bundle --answer-yes -target https://xx.xx.xxx.xx:8089 -auth admin:xxxmxmxm

hope @erw550  this will help to you.

[justynap_ldz]

Hello @erw550 , we have exactly the same issue.
Have you solved the issue by upgrading SHC to 9.0 or in any other way?

[erw550]

Hi,

There are new requirements for certificates in version 9.0. We hade to go back to version 8.1.6.

[richgalloway]

The SHC needs to be updated to match the deployer version.

The alternative is to separate the DS from the deployer and only upgrade the DS to 9.0.

[isoutamo]

Here is more information about security changes on splunk 9.0.0 https://docs.splunk.com/Documentation/Splunk/9.0.0/Security/Updates. Also this should read before updates https://lantern.splunk.com/Splunk_Platform/Product_Tips/Enterprise/Upgrading_Splunk_Enterprise

r. Ismo