Splunk Enterprise

Why is server certificate hostname validation disabled?

Gregski11
Contributor

I am trying to enable Server Certificate Hostname Validation in the server.conf file and I literally cut and pasted the command 

sslVerifyServerName = true # turns on TLS certificate host name validation

from the Splunk documentation and when I restart Splunk on this on prem deployment server it says :

WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

 

now I get the CLI command is cliVerifyServerName instead of sslVerifyServerName, but I even tried having both lines there and it still does not like it

I have issued an Enterprise web certificate to this server, it is still valid for two years, so I am at a total loss here please help

 

Labels (1)
Tags (1)

hrawat_splunk
Splunk Employee
Splunk Employee

Can you set following and re-test?

useSslClientSessionCache=true

 Although it's not related to certification validation, but it appears `sslVerifyServerName` is not working (defect) but likely to work with above config.

0 Karma

westy74
Engager

For me, applying the suggested fix of:
/opt/splunk/bin/splunk cmd openssl rsa -in myServerPrivateKey.key -out myServerPrivateKey.key

to remove password out of the private.key.key,

Got rid of the startup error message of:
WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

bansodesant
Explorer

These following steps worked for me to enable Splunk Web with custom certificates. If you are using third party certificates then make sure server key file is password less. You can verify it with following command (change key file name with your file name)

$SPLUNK_HOME/bin/splunk cmd openssl rsa -in mySplunkWebPrivateKey.key -text

 

Follow following steps to create and use your own certificates:

cd /opt/splunk/etc/auth
mkdir mycerts


Change to mycerts directory to create custom certificates : cd mycerts

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myCertAuthPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myCertAuthPrivateKey.key -out myCertAuthCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myCertAuthCertificate.csr -sha512 -signkey myCertAuthPrivateKey.key -CAcreateserial -out myCertAuthCertificate.pem -days 1095

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myServerPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myServerPrivateKey.key -out myServerCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myServerCertificate.csr -SHA256 -CA myCertAuthCertificate.pem -CAkey myCertAuthPrivateKey.key -CAcreateserial -out myServerCertificate.pem -days 1095

Remove password from key file using following command as splunk version 9.0 doesn't support key file with passowrd

/opt/splunk/bin/splunk cmd openssl rsa -in myServerPrivateKey.key -out myServerPrivateKey.key

cat myServerCertificate.pem myServerPrivateKey.key myCertAuthCertificate.pem > mySplunkWebCertificate.pem

vi /opt/splunk/etc/system/local/web.conf

[settings]
enableSplunkWebSSL = true
privKeyPath = /opt/splunk/etc/auth/mycerts/myServerPrivateKey.key
serverCert = /opt/splunk/etc/auth/mycerts/mySplunkWebCertificate.pem

Tags (1)

norbertt911
Path Finder

Hi,

Your process is correct, but the topic is not about this.  You just describe how to use a custom/third-party SSL certificate for the web GUI, but cliVerifyServerName is different from that.

 

0 Karma

tsullivan06
Explorer

@Gregski11 ,

I've been (painfully) working through some of the 'new SSL' processes myself. Unfortunately the documentation is as helpful as I was hoping and in some cases is actually functionally incorrect.  However I have gotten what I believe to be the correct web.conf config done, server.conf is proving to have a few extra gremlins current.

What I suspect is happening here is that you're providing the server PEM you got from the CA but what Splunk's looking really wanting here is a  'combined' cert:  

https://docs.splunk.com/Documentation/Splunk/9.0.0/Security/HowtoprepareyoursignedcertificatesforSpl...

I've had to add a little bit more than what the docs call out, probably because I created the CA cert and the server cert on the Splunk instance but here's my web.conf file looks like:

 

[settings]
enableSplunkWebSSL = true
sslPassword = $7$7mytLzGbDj/xZXYnAuZCXks/FYVEAi7AqKXHOqezXPBM1qwfsHRPM8Fe
privKeyPath = /opt/splunk/etc/auth/mycerts/myServerPrivateKey.key
serverCert = /opt/splunk/etc/auth/mycerts/myServerCert.pem
sslRootCAPath = /opt/splunk/etc/auth/mycerts/myCertAuthCertificate.pem

 

tools.sessions.timeout = 8640

Hope it helps!

 

Gregski11
Contributor

alright this is really odd when I simply point the server.conf file at the location of our existing SSL certificate (which we obtained from our very own Microsoft Certificate Authority)  with these two simple lines:

[sslConfig]
serverCert = mycerts\mySplunkWebCertificate.pem

 

Instead of using that certificate Splunk tries to generate a self signed one??? with an error like the one below that we caught while having entered an invalid path to our cert by mistake

"The certificate generation script did not generate the expected certificate"

 

when the path is correct it creates an empty PFX file in that directory called mySplunkWebCertificate.pem.pfx 

what is it trying to do?  why is it creating a file, it should just simply use our existing cert

has anybody gotten this to work?

0 Karma

norbertt911
Path Finder

I added the cliVerifyServerName = true stanza to [sslConfig] and the result is:

ERROR: certificate validation: self signed certificate in certificate chain - endless flood and the server won't start at all.  I using a wildcard SSL certificate issued by a CA, so there is no way to have self-signed chains - or it's checking with the default certificate maybe.

Conclusion: I will wait for the fix from Splunk's side (as for the federated.conf issue or the Python upgrade readiness app issue)

 

Gregski11
Contributor

@norbertt911 wrote:

I added the cliVerifyServerName = true stanza to [sslConfig] and the result is:

ERROR: certificate validation: self signed certificate in certificate chain - endless flood and the server won't start at all.  I using a wildcard SSL certificate issued by a CA, so there is no way to have self-signed chains - or it's checking with the default certificate maybe.

Conclusion: I will wait for the fix from Splunk's side (as for the federated.conf issue or the Python upgrade readiness app issue)

 


thank you so much for doing that, I get the same thing, I am going to open a support case with Splunk for this one

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try moving the comment in server.conf (Splunk doesn't support trailing comments).

# turns on TLS certificate host name validation
sslVerifyServerName = true 

If it works, then submit feedback on the documentation page on which you found the setting.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Gregski11
Contributor

oh wow, that sucks, wish they didn't do that in their examples

so here's all I have in my SSL Config stanza now and it still borx

 

[sslConfig]
sslVerifyServerCert = true
sslVerifyServerName = true

 

WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have you tried the suggestion in the message?

[sslConfig]
cliVerifyServerName = true
---
If this reply helps you, Karma would be appreciated.
0 Karma

Gregski11
Contributor

@richgalloway wrote:

Have you tried the suggestion in the message?

 

[sslConfig]
cliVerifyServerName = true

 


yes I have, it did not get rid of the WARNING, I also read further in the Splunk documentation to edit the web.config file with the same lines so I did, still no dice, I am tempted to open a Splunk Support case

is TLS working for you?

 

Tags (1)
0 Karma

computermathguy
Explorer

Did Splunk have any recommendations? I'm having the same issue.

0 Karma

chingtawang
New Member

reference document

splunk version : 9.0.2

# turns on TLS certificate host name validation
cliVerifyServerName = true
# Reference the file that contains all root certificate authority certificates combined together
sslRootCAPath = /splunk/opt/splunk/etc/auth/test/new.pem

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I haven't gotten it to work, but haven't tried real hard, either.

Please let us know what Support says.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Gregski11
Contributor

so just noticed a major change in SSL certificate request process starting with version 9.0.0 looks like Splunk has gotten rid of the: 

"Remove the password from the private key. You must do this because Splunk Web does not support private key passwords."

step, so does this mean Splunk Web now supports private key passwords? very interesting 

0 Karma

bansodesant
Explorer

These following steps worked for me to enable Splunk Web with custom certificates. If you are using third party certificates then make sure server key file is password less. You can verify it with following command (change key file name with your file name)

$SPLUNK_HOME/bin/splunk cmd openssl rsa -in mySplunkWebPrivateKey.key -text

 

Follow following steps to create and use your own certificates:

cd /opt/splunk/etc/auth
mkdir mycerts


Change to mycerts directory to create custom certificates : cd mycerts

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myCertAuthPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myCertAuthPrivateKey.key -out myCertAuthCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myCertAuthCertificate.csr -sha512 -signkey myCertAuthPrivateKey.key -CAcreateserial -out myCertAuthCertificate.pem -days 1095

/opt/splunk/bin/splunk cmd openssl genrsa -aes256 -out myServerPrivateKey.key 2048

/opt/splunk/bin/splunk cmd openssl req -new -key myServerPrivateKey.key -out myServerCertificate.csr

/opt/splunk/bin/splunk cmd openssl x509 -req -in myServerCertificate.csr -SHA256 -CA myCertAuthCertificate.pem -CAkey myCertAuthPrivateKey.key -CAcreateserial -out myServerCertificate.pem -days 1095

Remove password from key file using following command as splunk version 9.0 doesn't support key file with passowrd

/opt/splunk/bin/splunk cmd openssl rsa -in myServerPrivateKey.key -out myServerPrivateKey.key

cat myServerCertificate.pem myServerPrivateKey.key myCertAuthCertificate.pem > mySplunkWebCertificate.pem

vi /opt/splunk/etc/system/local/web.conf

[settings]
enableSplunkWebSSL = true
privKeyPath = /opt/splunk/etc/auth/mycerts/myServerPrivateKey.key
serverCert = /opt/splunk/etc/auth/mycerts/mySplunkWebCertificate.pem

Tags (1)
0 Karma

adnanhakiim
Loves-to-Learn Lots

 

Hi,
I have an issue after upgrading the Splunk Enterprise version to the latest version (9.0.4.1), once I upgraded the Splunk I got a warning alert below:

 

WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details

 

Then I configured the cliVerifyServerName as suggested.
 
[sslConfig] cliVerifyServerName=true
sslRootCAPath=/opt/splunk/etc/auth/splunkweb/ourcertificate.crt

 

But after I restarted the splunkd and try to run splunk show shcluster-status then I got an error below.

 

ERROR: certificate validation: self signed certificate in certificate chain Encountered some errors while trying to obtain shcluster status. Couldn't complete HTTP request: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
 
What do I need to solve the issue?
Thanks.
0 Karma

norbertt911
Path Finder

Hi,

As far as I understand the root problem of this issue that Splunk cannot determinate that your SSL certificate issuer is trustable or not.

I play

ed a lot with this - I using CA trusted wildcard certificate. And end up this configuration in server.conf:

sslVerifyServerCert = true
cliVerifyServerName = true
serverCert = $SPLUNK_HOME/etc/auth/mycert/cert-with-key.pem (-> servercert+middle-chain cert+root cert+ private key)
sslRootCAPath = /etc/ssl/certs/ca-bundle.crt

sslRootCAPath is the path of your OS trusted CA bundle. You may need to add Your issuer to this list manually. (the root cert only). Depending by OS, but same process:
https://ubuntu.com/server/docs/security-trust-store 

Now I have no such warning, and seems everything working fine. (May could work if you pointing the your root cert only with sslRootCAPath, but that not tested )

KR.

 

 

 

 

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...