Security

Having Difficulty with SSL certs - Splunk 8 on prem

BrianS
Observer

Hello, and thanks for reading this.

Having issues with securing the web site for our on-prem Splunk 8 Enterprise instance. This is a vanilla install at this point, so little customization has taken place. The Splunk web site works fine without a cert. At the moment, we are trying to simply get the web site restricted to TLS 1.2 traffic only using a third-party certificate - in the future, we will look at other facets of this. I tried following the steps found in the "Securing the Splunk Platform" document (https://docs.splunk.com/Documentation/Splunk/8.0.5/Security/WhatyoucansecurewithSplunk).

Work Log:

1. Requested and downloaded a cert from a 3rd party organization, trusted in our domain.  Imported the cert into the server (Windows Server 2016, if it matters) to complete the enrollment process.

2. Exported the cert as a PFX file, including the private key. Exported the Intermediate CA cert as a CER file. Exported the Root CA cert as a CERT file.

3. Opened an admin PowerShell window, and navigated to "$SPLUNK_HOME/bin/" Ran the following (filenames and paths are place holders):
.\splunk.exe cmd openssl pkcs12 -in C:\certs\SSL.pfx -nocerts -out C:\certs\SSL_key.pem -nodes
.\splunk.exe cmd openssl pkcs12 -in C:\certs\SSL.pfx -nokeys -out C:\certs\SSL_cert.pem -nodes
.\splunk.exe cmd openssl x509 -in C:\certs\Int_CA.cer -out C:\certs\Int_CA_cert.pem
.\splunk.exe cmd openssl x509 -in C:\certs\Root_CA.cer -out C:\certs\Root_CA_cert.pem

4. Using Notepad, I opened the SSL_Cert.pem, Int_CA_cert.pen, and Root_CA_cert.pem files, and I copied and pasted the contents from the BEGIN CERTIFICATE line to the END CERTIFICATE line, combining them into a single PEM file (let's call it SSL_combined.pem) like so:
-----BEGIN CERTIFICATE-----
<SSL Certificate>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<Intermediate CA Certificate>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<Root CA Certificate>
-----END CERTIFICATE-----

5. Now that the combined certs and key were in PEM format, I created a folder for them at "$SPLUNK_HOME/etc/mycerts/" and copied them there.

6. I edited the "$SPLUNK_HOME/etc/system/local/web.conf" file as follows:
Under the [settings] section, I changed the value of enableSplunkWebSSL from false to true.
I added a line which read privKeyPath = /home/etc/auth/mycerts/SSL_key.pem
I added a line which read serverCert = /home/etc/auth/mycerts/SSL_combined.pem
I changed the value of sslVersions from tls to tls1.2

7. Finally, I restarted the Splunk services by running ".\splunk.exe restart splunkd" which completes with no errors.

However, when we try to open the Splunk web page, the browser hangs at "Performing TLS Handshake" in Firefox. In Chrome, it fails with an ERR_TIMED_OUT message. In IE 11, the browser simply hangs up with no error.


Captured log in Firefox of connection attempt, but I never see any connection get established. There is an attempt to connect, which times out. 

Any idea which direction to go from here?

Labels (1)
0 Karma

thambisetty
SplunkTrust
SplunkTrust

make sure you are not setting up key while generating CSR. Splunk web can't open privatekey if its set with key.

public.pem should be in below order:
server signed cert
intermediate cert
root cert

PrivKey.key is the one generated before generating CSR and the PrivKey.key should not be set with any secret key.

[settings]
enableSplunkWebSSL = true

privKeyPath = $SPLUNK_HOME/etc/auth/mycerts/PrivKey.key
serverCert = $SPLUNK_HOME/etc/auth/mycerts/public.pem

 

————————————
If this helps, give a like below.
0 Karma

BrianS
Observer

The order on my combined cert (public.pem) checks out with the one you presented.   So, that is probably not the issue.

The key file might be the problem.  As noted above, I tried to extract it from the PFX (using OpenSSL, as above) which included the private key.  I guess that doesn't work?  The contents of the extracted private key file look like this (edited, of course):

Bag Attributes
Microsoft Local Key set: <No Values>
localKeyID: <numbers>
friendlyName: {<characters>}
Microsoft CSP Name: Microsoft Software Key Storage Provider
Key Attributes
X509v3 Key Usage: 90
-----BEGIN PRIVATE KEY-----
<characters>
-----END PRIVATE KEY-----

0 Karma

thambisetty
SplunkTrust
SplunkTrust

I believe it would have been worked if there was no key set to private key. In my opinion there is secret key set to privkey and that might be causing issue.

————————————
If this helps, give a like below.
0 Karma

BrianS
Observer

I commented out the privKeyPath line in web.conf

privKeyPath = /home/etc/auth/mycerts/name.key

 

Ran a splunk.exe restart splunkd from PowerShell, but change.  Page does not load in HTTPS.  Reset everything to HTTP so that I can use the site while I puzzle this out.

I suppose that I may need to open a support request on this.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...