Splunk with certbot SSL certificates

[jnilsson]

Hello,

I've spent probably 8+hrs now trying to debug how to get SSL certificates working with splunk web and finally got it working, so posting this here to hopefully help someone in the future.

Using these links as a reference:

https://docs.splunk.com/Documentation/Splunk/9.0.2/Security/Turnonbasicencryptionusingweb.conf

https://docs.splunk.com/Documentation/Splunk/9.0.2/Security/HowtoprepareyoursignedcertificatesforSpl...

The hardest part was figuring out how to use the certificates provided by certbot into a format that splunk recognizes.

The following steps ended up working:

1) Create /opt/splunk/etc/system/local/web.conf by copying /opt/splunk/etc/system/default/web.conf and change the line "enableSplunkWebSSL = false" to "enableSplunkWebSSL = true"

2) Install and configure certbot to obtain certificates as needed. They'll be in /etc/letsencrypt/live/$my_domain/ instead of /opt/splunk/etc/auth/splunkweb/ and they're not in a format that splunk can use.

3) The second link above gives some guidance on how to prepare the certbot certificates to the format that splunk needs them, which should be:

server certificate
private key
CA certificate

To do this, I'm creating the following certbot post renewal hook script:

/etc/letsencrypt/renewal-hooks/post/splunk.sh

#!/bin/bash

#change this my_domain variable to match the domain you are using
my_domain=XXXX
src_path=/etc/letsencrypt/live/$my_domain
dst_path=/opt/splunk/etc/auth/splunkweb

cat $src_path/cert.pem $src_path/privkey.pem $src_path/fullchain.pem > $dst_path/cert.pem
cat $src_path/privkey.pem > $dst_path/privkey.pem

chown splunk:splunk $dst_path/cert.pem $dst_path/privkey.pem
chmod 600 $dst_path/cert.pem $dst_path/privkey.pem

/opt/splunk/bin/splunk restart
#EOF

And make the script executable:

chmod +x /etc/letsencrypt/renewal-hooks/post/splunk.sh

4) Since you've already renewed the certificate with certbot, you can run the script directly:

/etc/letsencrypt/renewal-hooks/post/splunk.sh

The script should run automatically whenever certbot renews your certificate

[jnilsson]

I'm not sure what you are asking me to do. I tried to edit the message and I don't see any place to mark it as a question and answer.

Is there a different forum where I'm supposed to post solutions?

[richgalloway · Accepted solution]

There is no "make this a question and answer" button.  What I meant was to change the posting into a question (perhaps, "How do I use SSL with Splunk web?") and then reply to that question with the rest of the OP as an answer.  Then that reply could be accepted as the solution.

[jnilsson]

i think a human can see that the answer is in the OP. but i can mark your suggestion as the answer if that helps tick of a box somewhere so that this post isn't showing up as "needs an answer" still

[richgalloway]

Thanks for the hard work in solving the problem and for the great post.  Would you mind editing the post into question and answer?  Then the answer could be marked as a solution to better help those with similar problems find it in the future.