Why am I getting heavy forwarder error "TcpInputConfig - SSL server certificate not found, or password is wrong..."?

[briangmadden]

I need to send data from a security appliance to a Splunk Heavy Forwarder on a listening port using TCP-TLS. Getting the errors below everytime in opt/splunk/var/log/splunk/splunkd.log that Splunk is started.

ERROR    SSLCommon - Can't read key file    /opt/splunk/etc/certs/cert.pem    errno=151441516 error:0906D06C:PEM    routines:PEM_read_bio:no start line.
ERROR    TcpInputConfig - SSL server    certificate not found, or password is    wrong - SSL ports will not be opened
ERROR    TcpInputConfig - SSL context not    found. Will not open raw (SSL) IPv4    port 17814

Here are the steps I followed:

1. Generated CSR file on my Heavy Forwarder and sent to my certificate provider to have it signed.
2. Received *.cer back from my certificate provider.
3. Ran following command to convert *.cer into *.pem: openssl x509 -inform pem -in certificate.cer -outform der -out certificate.pem
4. Copied cert.pem & InternalRootCA.pem to /opt/splunk/etc/certs

5. Here is my inputs.conf

   [SSL]
   rootCA = $SPLUNK_HOME/etc/certs/InternalRootCA.pem
   serverCert = $SPLUNK_HOME/etc/certs/cert.pem
   password = ***************
   requireClientCert = false
   [tcp-ssl://17814]
   sourcetype = syslog
   index = **
   Restart Splunk & I get errors:

   ERROR SSLCommon - Can't read key file /opt/splunk/etc/certs/cert.pem errno=151441516 error:0906D06C:PEM routines:PEM_read_bio:no start line.
   ERROR TcpInputConfig - SSL server certificate not found, or password is wrong - SSL ports will not be opened
   ERROR TcpInputConfig - SSL context not found. Will not open raw (SSL) IPv4 port 17814
   The cert folder only includes the two files
   InternalRootCA.pem
   cert.pem

[jbrodsky_splunk]

The error "ERROR SSLCommon - Can't read key file /opt/splunk/etc/certs/cert.pem errno=151441516 error:0906D06C:PEM routines:PEM_read_bio:no start line." can be caused if you mistakenly swap the certificate path with the root CA path in the .conf file.

[jplumsdaine22]

Have you checked the file permissions on those certificate files? IE does the user Splunk runs as have permissions to read them? You can check the file contents with openssl and verify the keys etc with commands found here as well https://www.sslshopper.com/article-most-common-openssl-commands.html

[briangmadden]

I modified all the directory and file permissions to make splunk the owner.