Applying Certificates to Secure Port 8000 (web.conf) and 8089 (server.conf) in v8.2*- Solution

[amt]

I had so much trouble with this but figured I would share what I did to make it work for me. You may have other ways of doing it but I found very little guidance online to help someone going through the process. If you have done other things that worked for you, feel free to reply and share.

[tringener]

From the post I'm assuming the install is installed on the default C: drive? What changes to this process need to change if Splunk was installed on a different drive?

Several commands are using environment variables and I don't see any on the server a colleague previously installed Splunk 9.0.x on. Do I need to manually add system variables or should the installer have done that during the install? If have need to add them what are they, where do they get installed (user or system), where do they point too? 

[isoutamo]

Usually it’s best to create a new question instead of add question to old solved one! 
Here is one conf presentation about TLS stuff https://conf.splunk.com/files/2023/slides/SEC1936B.pdf

[amt · Accepted solution]

SECURING PORT 8000 (web.conf) and 8089 (server.conf) WITH CERTIFICATES

Port 8000 - web.conf
Putty into the target server
sudo as splunk
create new <new_namedir> directory under /opt/splunk/etc/auth/
# /opt/splunk/etc/auth/<new_namedir>
# cd /opt/splunk/bin/
# ./splunk cmd openssl genrsa -aes256 -out CustomName_SplunkWebPrivateKey.key 2048
To continue forward, remove password from key because splunk web does not support it.
# ./splunk cmd openssl rsa -in CustomName_SplunkWebPrivateKey.key -out CustomName_SplunkWebPrivateKey.key
To confirm the password has been removed, perform the following and see if you can see the unencrypted text in the certificate.
# openssl rsa -in CustomName_SplunkWebPrivateKey.key -text
Generate a new certificate signing request (CSR) using your private key file CustomName_SplunkWebPrivateKey.key:
# ./splunk cmd openssl req -new -key CustomName_SplunkWebPrivateKey.key -out CustomName_SplunkWebCert.csr
You will be prompted for a number of metadata attributes. You can skip all (if you want) by just hitting enter except for 'Common Name', which you will use the FQDN of the server.
Copy the CSR to a location on the server where you can access it via WinSCP. Download the file to your Desktop.
Send to the CA to request the following (Important: Ensure they are returned in .pem format):

1. Signed and returned CSR (Server Certificate)
2. Intermediate Certificate
3. Root Certificates

When the signed server certificate (CSR) along with the Intermediate and Root server are returned by the CA. The certificates must be concatenated.
Combine certificates in this order and save the .pem file (You can simply modify the server cert by adding the intermediate and root certificates as follows):
-----BEGIN CERTIFICATE-----
... (certificate for your SERVER CERTIFICATE)...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
... (the INTERMEDIATE certificate)...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
... (the ROOT certificate for the CA)...
-----END CERTIFICATE-----
Place the certificates in the following directory: /opt/splunk/etc/auth/<new_namedir>/ and ensure permissions on the certificates are for splunk:splunk
Navigate to the web.conf file to point it to the certificates
# cd /opt/splunk/etc/system/local/web.conf
Update the web.conf (port 8000) to include the following under the settings stanza:
[settings]
enableSplunkWebSSL = 1
privKeyPath = /opt/splunk/etc/auth/<new_namedir>/CustomName_SplunkWebPrivateKey.key
serverCert = /opt/splunk/etc/auth/<new_namedir>/FQDN.pem

Restart Splunk services
# sudo su - /opt/splunk/bin/splunk restart
Log into your Splunk instance on port 8000 and confirm certificate by clicking on the certificate icon to the left of your web address.

Port 8089 - server.conf
Generate a new CSR (steps to perform are above) and request a new signed server certificate along with Intermediate and Root certificates in .pem format.
Concatenate the signed server certificate as follows:
-----BEGIN CERTIFICATE-----
... (certificate for your SERVER CERTIFICATE)...
-----END CERTIFICATE-----
-----BEGIN ENCRYPTED PRIVATE KEY-----
... (the encrypted private key)...
-----END ENCRYPTED PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
... (the ROOT certificate for the CA)...
-----END CERTIFICATE-----
Place the .pem file in the /opt/splunk/etc/auth/<new_8089_dirname>
Update the server.conf under /opt/splunk/etc/system/local with the following under the [sslConfig] stanza
[sslConfig]
enableSplunkdSSL = true
sslRootCAPath = /opt/splunk/etc/auth/<new_8089_dirname>/RootCertificate.pem
serverCert = /opt/splunk/etc/auth/<new_8089_dirname>/ConcatenatedCertwEncryptedPrivKey.pem
sslPassword = <password> (Password gets hashed after saving)

Reboot Splunk and you should be able to login to the Splunk Web interface after.
To confirm 8089 is secured, you can place the following address in the web browser https://<FQDN_of_Server>:8089/ into the browser and check the certificate by clicking the certificate icon to the left of the web address.

Hope this helps!

[leighsatch]

you rock amt, this works a treat!