Server certificate renewal failed

[gc]

Hello there,

I have a problem with one of our Splunk installations on Windows. The server certificate is expired and I'm unable to renew it. I've tried renaming C:\Program Files\Splunk\etc\auth\server.pem and restarting Splunk, which ends with that:

The certificate generation script did not generate the expected certificate file:C:\Program Files\Splunk\etc\auth\server.pem. Splunkd port communication will not work.
SSL certificate generation failed.

And I also tried this command: C:\Program Files\Splunk\bin>splunk createssl server-cert -d "C:\Program Files\Splunk\etc\auth" -n server -c *servername*
Which also fails with this:

CreateProcess: error 193
Command failed (ret=-1), exiting.

Anyone knows how to fix this? Thanks in advance.
Best regards

Alex

[gc]

Hi,

anyone else with a suggestion? 😕
Thanks again, best regards

Alex

[deepakc]

Try this, not sure if it will work, but worth a try. 

See if the variable is pointing to this file which contains SSL config / library's etc 

echo %OPENSSL_CONF%

Set it as below and try again. 

set OPENSSL_CONF=c:\Program Files\Splunk\openssl.cnf

[gc]

Hi there,

thank you for your idea, but unfortunately it was not working:

The path is correct. Is there any way to find out, why the generation is failing? Checked some logs, but couldn't find anything that was helping...

[deepakc]

There may be something in splunkd.log(not sure) find this in $SPLUNK_HOME\var\log\splunk

Whats the output of this? (I'm starting to think the root cacert.pem has something to do with this.)

openssl x509 -in "c:\Program Files\Splunk\etc\auth\cacert.pem" -text -noout

Does it show its expired? may be this has something to do with it.

Try and rename that file cacert.pem or it could be ca.pem and do a restart

[gc]

I have checked the log, there is nothing there. In fact there is only 1 log with new entries. These are the last entries from splunkd-utility.log:

05-17-2024 16:44:40.570 +0200 INFO ServerConfig - Found no hostname options in server.conf. Will attempt to use default for now.
05-17-2024 16:44:40.570 +0200 INFO ServerConfig - Host name option is "".
05-17-2024 16:44:40.570 +0200 INFO ServerConfig - TLS Sidecar disabled
05-17-2024 16:44:40.570 +0200 WARN SSLOptions - server.conf/[sslConfig]/sslVerifyServerCert is false disabling certificate validation; must be set to "true" for increased security
05-17-2024 16:44:40.570 +0200 INFO ServerConfig - No 'C:\Program Files\Splunk\etc\auth\server.pem' certificate found. Splunkd communication will not work without this. If this is a fresh installation, this should be OK.
05-17-2024 16:44:40.586 +0200 INFO ServerConfig - disableSSLShutdown=0
05-17-2024 16:44:40.586 +0200 INFO ServerConfig - Setting search process to have long life span: enable_search_process_long_lifespan=1
05-17-2024 16:44:40.586 +0200 INFO ServerConfig - enableTeleportSupervisor=0, scsEvironment=production
05-17-2024 16:44:40.586 +0200 INFO ServerConfig - certificateStatusValidationMethod is not set, defaulting to none.
05-17-2024 16:44:40.586 +0200 INFO ServerConfig - Splunk is starting with EC-SSC disabled

cacert.pem is valid till 2027 and I have checked server.conf, which has no entry for hostname. But this seems to be normal, have checked against another installation.

[deepakc]

That WARN is just for extra security.

Its still having issues with the server.pem file 

I'm out of options to check mate, consider logging a support call, or you could if this is an option to you, backup /etc/apps folder and re-install Splunk,  and restore the backed up /etc/apps folder, I know this is a drastic step...but might be quicker. 

[gc · Accepted solution]

Thanks, I did a reinstallation. And just to be sure, I had to save + restore \var\lib\splunk
Thanks for your help, have a good day 🙂

[deepakc]

1. Check Your Admin Permissions etc 

2. Could it be AV / blocking the action - command?  

[gc]

Hello,

thanks for replying, checked the permission and disabled AV, still the same outcome. Any other ideas?

Best regards
Alex