unable to update SSL certificate -Cannot decrypt private key -Splunk 9.0.0

[saibargavg]

Hi Everyone,

We're in the process of updating the SSL certificates on our Splunk servers. However, when attempting the upgrade, we encounter the following error:

"Cannot decrypt private key in "/opt/splunk/etc/apps/*/local/Splunk.key" without a password. Network communication with splunkweb may fail or hang. Consider using an unencrypted private key for Splunkweb's SSL certificate."

Could anyone provide assistance with this issue?

Below are the steps we followed while generating the certificate. Please let us know if you spot any mistakes. We're running Splunk 9.0.0.

## Go to /root/certs/
cd /root/certs/

## Create new directory for the certs
mdkir certs_2024

## Create Server Key
openssl genrsa -des3 -out splunk.key 2048
password123######
password123######

## Create a No Pass Key
openssl rsa -in splunk.key -out splunk.nopass.key
enter passphrase - <<<password>>>

## Generate the csr file
openssl req -new -sha256 -key splunk.nopass.key -out splunk.csr

 once we get the certificate, we are running the below steps. 

vi end_entity_cert <<paste the end_entity_cert value for the hostname and save>>

vi intermediate_cert <<paste the intermediate_cert value for the hostname and save>>
cp splunk.nopass.key /opt/splunk/etc/apps/App_hostname_ssl/local

Go to certificates folder - cd /home/Splunk/certs_renewal/

Copy the rootCA.pem into /opt/splunk/etc/apps/App_hostname_ssl/local

## Create Certificate Chain

cat end_entity_cert splunk.key intermediate_cert rootCA.pem >>full.pem

## Verify Certificate Validity

openssl x509 -enddate -noout -in full.pem

./splunk restart

[PickleRick]

It's not all that's at play here but you're creating a whole lot of files (you could just create a key with -nodes option to have it non-encrypted) and your config apparently points to splunk.key which - judging by the sequence of commands - is encrypted.

As a side note - putting your private key into an app is not necessarily the most secure thing to do.