Why is server certificate hostname validation disabled?

[Gregski11]

I am trying to enable Server Certificate Hostname Validation in the server.conf file and I literally cut and pasted the command 

sslVerifyServerName = true # turns on TLS certificate host name validation

from the Splunk documentation and when I restart Splunk on this on prem deployment server it says :

WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

now I get the CLI command is cliVerifyServerName instead of sslVerifyServerName, but I even tried having both lines there and it still does not like it

I have issued an Enterprise web certificate to this server, it is still valid for two years, so I am at a total loss here please help

[hrawat_splunk]

Can you set following and re-test?

useSslClientSessionCache=true

 Although it's not related to certification validation, but it appears `sslVerifyServerName` is not working (defect) but likely to work with above config.

[westy74]

For me, applying the suggested fix of:
/opt/splunk/bin/splunk cmd openssl rsa -in myServerPrivateKey.key -out myServerPrivateKey.key

to remove password out of the private.key.key,

Got rid of the startup error message of:
WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

[bansodesant]

These following steps worked for me to enable Splunk Web with custom certificates. If you are using third party certificates then make sure server key file is password less. You can verify it with following command (change key file name with your file name)

$SPLUNK_HOME/bin/splunk cmd openssl rsa -in mySplunkWebPrivateKey.key -text

Follow following steps to create and use your own certificates:

cd /opt/splunk/etc/auth
mkdir mycerts

Change to mycerts directory to create custom certificates : cd mycerts

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myCertAuthPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myCertAuthPrivateKey.key -out myCertAuthCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myCertAuthCertificate.csr -sha512 -signkey myCertAuthPrivateKey.key -CAcreateserial -out myCertAuthCertificate.pem -days 1095

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myServerPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myServerPrivateKey.key -out myServerCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myServerCertificate.csr -SHA256 -CA myCertAuthCertificate.pem -CAkey myCertAuthPrivateKey.key -CAcreateserial -out myServerCertificate.pem -days 1095

Remove password from key file using following command as splunk version 9.0 doesn't support key file with passowrd

/opt/splunk/bin/splunk cmd openssl rsa -in myServerPrivateKey.key -out myServerPrivateKey.key

cat myServerCertificate.pem myServerPrivateKey.key myCertAuthCertificate.pem > mySplunkWebCertificate.pem

vi /opt/splunk/etc/system/local/web.conf

[settings]
enableSplunkWebSSL = true
privKeyPath = /opt/splunk/etc/auth/mycerts/myServerPrivateKey.key
serverCert = /opt/splunk/etc/auth/mycerts/mySplunkWebCertificate.pem

[norbertt911]

Hi,

Your process is correct, but the topic is not about this.  You just describe how to use a custom/third-party SSL certificate for the web GUI, but cliVerifyServerName is different from that.

[tsullivan06]

@Gregski11 ,

I've been (painfully) working through some of the 'new SSL' processes myself. Unfortunately the documentation is as helpful as I was hoping and in some cases is actually functionally incorrect.  However I have gotten what I believe to be the correct web.conf config done, server.conf is proving to have a few extra gremlins current.

What I suspect is happening here is that you're providing the server PEM you got from the CA but what Splunk's looking really wanting here is a  'combined' cert:  

https://docs.splunk.com/Documentation/Splunk/9.0.0/Security/HowtoprepareyoursignedcertificatesforSpl.... 

I've had to add a little bit more than what the docs call out, probably because I created the CA cert and the server cert on the Splunk instance but here's my web.conf file looks like:

[settings]
enableSplunkWebSSL = true
sslPassword = $7$7mytLzGbDj/xZXYnAuZCXks/FYVEAi7AqKXHOqezXPBM1qwfsHRPM8Fe
privKeyPath = /opt/splunk/etc/auth/mycerts/myServerPrivateKey.key
serverCert = /opt/splunk/etc/auth/mycerts/myServerCert.pem
sslRootCAPath = /opt/splunk/etc/auth/mycerts/myCertAuthCertificate.pem

tools.sessions.timeout = 8640

Hope it helps!

[Gregski11]

alright this is really odd when I simply point the server.conf file at the location of our existing SSL certificate (which we obtained from our very own Microsoft Certificate Authority)  with these two simple lines:

[sslConfig]
serverCert = mycerts\mySplunkWebCertificate.pem

Instead of using that certificate Splunk tries to generate a self signed one??? with an error like the one below that we caught while having entered an invalid path to our cert by mistake

"The certificate generation script did not generate the expected certificate"

when the path is correct it creates an empty PFX file in that directory called mySplunkWebCertificate.pem.pfx 

what is it trying to do?  why is it creating a file, it should just simply use our existing cert

has anybody gotten this to work?

[norbertt911]

I added the cliVerifyServerName = true stanza to [sslConfig] and the result is:

ERROR: certificate validation: self signed certificate in certificate chain - endless flood and the server won't start at all.  I using a wildcard SSL certificate issued by a CA, so there is no way to have self-signed chains - or it's checking with the default certificate maybe.

Conclusion: I will wait for the fix from Splunk's side (as for the federated.conf issue or the Python upgrade readiness app issue)

[Gregski11]

---

@norbertt911 wrote:

I added the cliVerifyServerName = true stanza to [sslConfig] and the result is:

ERROR: certificate validation: self signed certificate in certificate chain - endless flood and the server won't start at all.  I using a wildcard SSL certificate issued by a CA, so there is no way to have self-signed chains - or it's checking with the default certificate maybe.

Conclusion: I will wait for the fix from Splunk's side (as for the federated.conf issue or the Python upgrade readiness app issue)

---

thank you so much for doing that, I get the same thing, I am going to open a support case with Splunk for this one

[richgalloway]

Try moving the comment in server.conf (Splunk doesn't support trailing comments).

# turns on TLS certificate host name validation
sslVerifyServerName = true 

If it works, then submit feedback on the documentation page on which you found the setting.

[Gregski11]

oh wow, that sucks, wish they didn't do that in their examples

so here's all I have in my SSL Config stanza now and it still borx

[sslConfig]
sslVerifyServerCert = true
sslVerifyServerName = true

WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

[richgalloway]

Have you tried the suggestion in the message?

[sslConfig]
cliVerifyServerName = true

[Gregski11]

---

@richgalloway wrote:

Have you tried the suggestion in the message?

[sslConfig]
cliVerifyServerName = true

---

yes I have, it did not get rid of the WARNING, I also read further in the Splunk documentation to edit the web.config file with the same lines so I did, still no dice, I am tempted to open a Splunk Support case

is TLS working for you?

[computermathguy]

Did Splunk have any recommendations? I'm having the same issue.

[chingtawang]

reference document

splunk version : 9.0.2

# turns on TLS certificate host name validation
cliVerifyServerName = true
# Reference the file that contains all root certificate authority certificates combined together
sslRootCAPath = /splunk/opt/splunk/etc/auth/test/new.pem

[richgalloway]

I haven't gotten it to work, but haven't tried real hard, either.

Please let us know what Support says.

[Gregski11]

so just noticed a major change in SSL certificate request process starting with version 9.0.0 looks like Splunk has gotten rid of the: 

"Remove the password from the private key. You must do this because Splunk Web does not support private key passwords."

step, so does this mean Splunk Web now supports private key passwords? very interesting 

[bansodesant]

These following steps worked for me to enable Splunk Web with custom certificates. If you are using third party certificates then make sure server key file is password less. You can verify it with following command (change key file name with your file name)

$SPLUNK_HOME/bin/splunk cmd openssl rsa -in mySplunkWebPrivateKey.key -text

Follow following steps to create and use your own certificates:

cd /opt/splunk/etc/auth
mkdir mycerts

Change to mycerts directory to create custom certificates : cd mycerts

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myCertAuthPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myCertAuthPrivateKey.key -out myCertAuthCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myCertAuthCertificate.csr -sha512 -signkey myCertAuthPrivateKey.key -CAcreateserial -out myCertAuthCertificate.pem -days 1095

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myServerPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myServerPrivateKey.key -out myServerCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myServerCertificate.csr -SHA256 -CA myCertAuthCertificate.pem -CAkey myCertAuthPrivateKey.key -CAcreateserial -out myServerCertificate.pem -days 1095

Remove password from key file using following command as splunk version 9.0 doesn't support key file with passowrd

/opt/splunk/bin/splunk cmd openssl rsa -in myServerPrivateKey.key -out myServerPrivateKey.key

cat myServerCertificate.pem myServerPrivateKey.key myCertAuthCertificate.pem > mySplunkWebCertificate.pem

vi /opt/splunk/etc/system/local/web.conf

[settings]
enableSplunkWebSSL = true
privKeyPath = /opt/splunk/etc/auth/mycerts/myServerPrivateKey.key
serverCert = /opt/splunk/etc/auth/mycerts/mySplunkWebCertificate.pem

[adnanhakiim]

Hi,

I have an issue after upgrading the Splunk Enterprise version to the latest version (9.0.4.1), once I upgraded the Splunk I got a warning alert below:

WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details

Then I configured the cliVerifyServerName as suggested.

[sslConfig] cliVerifyServerName=true

sslRootCAPath=/opt/splunk/etc/auth/splunkweb/ourcertificate.crt

But after I restarted the splunkd and try to run splunk show shcluster-status then I got an error below.

ERROR: certificate validation: self signed certificate in certificate chain Encountered some errors while trying to obtain shcluster status. Couldn't complete HTTP request: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.

What do I need to solve the issue?

Thanks.

[norbertt911]

Hi,

As far as I understand the root problem of this issue that Splunk cannot determinate that your SSL certificate issuer is trustable or not.

I play

ed a lot with this - I using CA trusted wildcard certificate. And end up this configuration in server.conf:

sslVerifyServerCert = true
cliVerifyServerName = true
serverCert = $SPLUNK_HOME/etc/auth/mycert/cert-with-key.pem (-> servercert+middle-chain cert+root cert+ private key)
sslRootCAPath = /etc/ssl/certs/ca-bundle.crt

sslRootCAPath is the path of your OS trusted CA bundle. You may need to add Your issuer to this list manually. (the root cert only). Depending by OS, but same process:
https://ubuntu.com/server/docs/security-trust-store 

Now I have no such warning, and seems everything working fine. (May could work if you pointing the your root cert only with sslRootCAPath, but that not tested )

KR.