Currently seeing issues after performing a certificate renewal.
Errors seen in splunkd.log
08-24-2022 00:58:03.942 +0000 ERROR SSLCommon - Can't read key file /opt/splunk/etc/auth/splunkweb/private.key errno=185073780 error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch.
08-24-2022 00:58:03.942 +0000 ERROR HTTPServer - SSL context could not be created - error in cert or password is wrong
08-24-2022 00:58:03.942 +0000 ERROR HTTPServer - SSL will not be enabled
The configuration for web.conf was validated in
Validated config in $SPLUNK_HOME/var/run/splunk/merged/web.conf and $SPLUNK_HOME /etc/system/local/web.conf
sslPassword = <HASHED_PASSWORD>
serverCert = $SPLUNK_HOME/etc/auth/splunkweb/server.pem
privKeyPath = $SPLUNK_HOME/etc/auth/splunkweb/private.key
I confirmed that the sslPassword is valid by decrypting the password using
/opt/splunk/bin/splunk show-decrypted --value <HASHED_PASSWORD>
openssl rsa -in /opt/splunk/etc/auth/splunkweb/private.key -noout -text
<decripted_HASHED_PASSWORD>
The private key opens correctly
The following commands were run to validate the integrity of certificates
openssl x509 -noout -modulus -in /opt/splunk/etc/auth/splunkweb/cert.pem | openssl md5
openssl x509 -noout -modulus -in /opt/splunk/etc/auth/server.pem | openssl md5
openssl rsa -noout -modulus -in /opt/splunk/etc/auth/splunkweb/private.key | openssl md5
All Values are the same
Host has been rebooted recently and selinux is disabled
