I am developing a .NET application that uses Splunk.Client (https://github.com/splunk/splunk-sdk-csharp-pcl/tree/master/src/Splunk.Client) to query a Splunk instance.
Under the hood, Splunk.Client makes a REST API call to splunkd.
The HTTP request contains a Splunk query (e.g.
index=main earliest=-1h), and splunkd responds with query results.
This works fine on our production environment.
For development and testing purposes, I installed a free edition of Splunk on my development PC (which, by the way, runs on Windows 10).
I configured the .NET application on my development PC to query my local Splunk instance.
Splunk comes out of the box with self-signed certificates.
Obviously these are not trusted by my computer, so Splunk.Client will throw an exception:
System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
I have little experience with certificates. I could just turn off certificate validation in the client (https://stackoverflow.com/questions/12506575/how-to-ignore-the-certificate-check-when-ssl), but I don't like the idea of a backdoor in production code.
I could go and buy a certificate, but that feels like overkill since it's just for development purposes.
I could make my own self-signed certificates, but that doesn't seem to bring any benefit over Splunk's self-signed certificates. There is a help document (https://docs.splunk.com/Documentation/Splunk/6.5.0/Security/Howtoself-signcertificates), but it's a dead end. The 'next steps' section has no reference to an article that explains how to make splunkd (port 8089) use the certificate. I edited Splunk's configuration files, but it seemed to have no effect. On this site I found questions concerning this issue, but hardly any useful feedback.
Splunk's self-signed certificate is good enough for me. What are the steps necessary to make my computer trust that certificate?
Disclaimer: I "don't have enough karma points to post links", hence the non-hyperlinks.
I found an answer myself before posting the question. I posted it anyway; maybe somebody facing the same problem will find it useful.
The answer is based on Windows 10; much of this will be different in Linux, obviously.
Trouble is, the whole certificate validation is obscure by design, so you won't know what you've done wrong until you've fixed all issues and the validation succeeds. I hope the following solution covers it all, but I can't be sure. While struggling to get things done, I may have changed something not mentioned here, simply because it seemed irrelevant at the time.
C:\Program Files\Splunk\etc\auth, there are two relevant certificates:
server.pem: this appears to be the certificate used by splunkd
ca.pem: this appears to be a root certificate that is necessary to get the certificate chain complete
Import both certificates into your computer's certificate store. Every certificate vendor will tell you how to. For Windows 10 instructions, just google 'mmc import certificate'.
There are a few pitfalls here:
Look at the details of Splunk's self-signed certificate. You can either do this with a web browser (navigate to https://localhost:8089 and drill through the security warnings until you get to see the certificate) or from command line:
"C:\Program Files\Splunk\bin\splunk.exe" cmd openssl s_client -connect localhost:8089
Notice the certificate is issued to the following common name (CN): SplunkServerDefaultCert.
It is possible to let your machine treat SplunkServerDefaultCert as an alias of localhost. The easiest way to accomplish this is to edit your hosts file. In Windows 10, this file is typically located in this folder:
Open the file in a text editor (e.g. Notepad++) and add the following line:
Note: the editor must be running as administrator, otherwise you will not be able to save your changes.
In a web browser, verify the URL works: https://SplunkServerDefaultCert:8089
-- I was setting up HTTPs for Splunk and was struggling with understanding if i setup my CA's incorrectly, when it turned out the issue that was resulting in (https yellow lock), was the CN. You helped a lot and I thank you so much for your great knowledge and understanding in troubleshooting this issue!