Implementing Bi-directional SSL on Splunkd with 3rd Party CA

[schmoud]

For my Splunk application I am required to implement bi-directional SSL using client and server certs on the Splunkd server with the intent of using the REST API. As an initial test I got one way ssl to work by following this as a rough guide, even though it is for Splunk Web. I am trying to just get it working in the browser (Firefox) before moving onto my custom application.

http://www.splunk.com/wiki/Community:SplunkWeb_SSL_3rdPartyCA

I added to my $SPLUNK_HOME/etc/system/local/server.conf under the [sslConfig] stanza

caCertFile = [pem file of for CA's public key]
sslKeysFile = [my concatenated key file]

-----BEGIN CERTIFICATE-----

[signed public key of server cert received from CA]

-----END CERTIFICATE-----

-----BEGIN RSA PRIVATE KEY-----

[private key of server cert]

-----END RSA PRIVATE KEY-----

-----BEGIN CERTIFICATE-----

[public key of CA cert]

-----END CERTIFICATE-----

sslKeysFilePassword = [splunk encrypted password]

All certificate files are in $SPLUNK_HOME/etc/auth/

I have set up a test CA on a separate machine where I create and sign certificates using OpenSSL.

One way SSL worked fine with this setup.

I added the requireClientCert = true to the server.conf file as well as generating a client certificate signed by the same CA with similar procedures to the ones used to create the server cert, this time creating a .pfx cert for browser installation.

Now when trying to access https://[splunkserverip]:8089 I get the option to pick my client cert (i have generated a couple client certs) and each time after I pick the client cert I have installed in the browser I get:

Error loading stylesheet: An unknown error has occurred (804b0014)
https://[splunkserverip]:8089/static/atom.xsl

and in the splunkd.log I see 10 repetitions of for ports 55565 - 55574

ERROR TcpInputFd - SSL Error = error:140D9115:SSL routines:SSL_GET_PREV_SESSION:session id context uninitialized

ERROR TcpInputFd - ACCEPT_RESULT=-1 VERIFY_RESULT=0

ERROR TcpInputFd - SSL Error for fd from HOST:[host] IP:[ip] PORT:[port]

Any references, suggestions, debugging methods, or solutions would be appreciated!

[gkanapathy · Accepted solution]

well, this may or may not be a bug, or it may be an artifact in the browser, or even just a bug in the broswer, but it shouldn't make any difference to you API access if you can get a URLs under /services/* without error. Chrome and Firefox try to fetch the stylesheets, but you might be able to disable that in the browser. If not, you can turn it off in Splunk with the atomFeedStylesheet = none in server.conf.