Why does Splunk fails to start with error message "The certificate generation script did not generate the expected certificate file"?

[dubeysantosh]

Why does Splunk service fails to start with error message: " The certificate generation script did not generate the expected certificate file"

I tried to start the splunkd service but the below messages were shown. Please advise how to fix it. Thanks.

[splunk@ bin]$ ./splunk start

Splunk> All batbelt. No tights.

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking configuration... Done.
The certificate generation script did not generate the expected certificate file:/opt/splunk/etc/apps/certs/myNewServerCertificate.pem. Splunkd port communication will not work.
SSL certificate generation failed.

[dubeysantosh]

I tested in lab and I was able to reproduce the issue and fix it. Please find the details below:

1. I moved myServerCertificate.pem to myServerCertificate.pem.orig
   [root@ selfsign]# ls -l myServerCertificate.pem
   ls: cannot access myServerCertificate.pem: No such file or directory
   [root@ selfsign]# ls -l myServerCertificate.pem*
   -rw-r--r-- 1 root root 4469 May 30 11:38 myServerCertificate.pem.orig

2. You can check location of ServerCertificate by running btool.
   [root@ selfsign]# splunk btool inputs list --debug | grep myServerCertificate.pem
   /opt/splunk/etc/system/local/inputs.conf serverCert = /opt/splunk/etc/auth/selfsign/myServerCertificate.pem

3. As you can see the myServerCertificate.pem doesn't exist.

[root@ selfsign]# ls -l myServerCertificate.pem
ls: cannot access myServerCertificate.pem: No such file or directory

4.I got the same error message "SSL certificate generation failed"

[root@ selfsign]# splunk restart
splunkd is not running. [FAILED]

Splunk> All batbelt. No tights.

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8090]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
The certificate generation script did not generate the expected certificate file:/opt/splunk/etc/auth/selfsign/myServerCertificate.pem. Splunkd port communication will not work.
SSL certificate generation failed.
[root@ selfsign]#

[nmohammed]

@dubeysantosh

were you able to fix it ? I see the same problem , when trying to add new search heads to a existing cluster .

I was able to resolve it, by manually copying the server.pem file in SPLUNK_HOME/etc/auth to SPLUNK_HOME/etc/auth/splunkweb and start Splunk.