Http Event Collector (HEC): SSL Self Signed Certificate Error

[marketplace]

Hi all,

I am currently testing the Http Event Collector (HEC) with a Splunk Cloud trial account. All I do is post data to the HEC url, and It works perfectly for a local instance for an Enterprise account at http://127.0.0.1:8088/services/collector/event

A solution I saw on the community forum was to disable the SSL validation. However, this isn't the best option to use in production for security reasons. Another Solution I saw was to upload certificates but this option isn't suited for a SaaS solution with many different customers.

Is it possible to solve this issue in a different way? And I would also like to ask if this problem would persist for normal production client accounts and along with a generic solution for it?  

Curl requests

curl https://prd-p-xxxxx.splunkcloud.com:8088/services/collector/event -H "Authorization: Splunk token" -d '{"event": "hello world"}'

Curl Response

curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.

Thank you for your time and assistance in addressing these inquiries. 

[tkopchak]

Can you post the output of this command? (replace with your trial stack's name). 

openssl s_client -connect prd-p-xxxxx.splunkcloud.com:8088

 I suspect the cert you'll see returned is from the Splunk internal CA, and that the Splunk Cloud trials are not set up with a signed cert on port 8089.

On a production/paid Splunk Cloud stack you'd send logs to https://http-inputs-<stack_name> .splunkcloud.com on port 443 and I've never seen an issue with certificate validation in those environments (it uses the same cert as the web interface). 

[marketplace]

Here is the response: 

CONNECTED(00000005)
depth=1 C = US, ST = CA, L = San Francisco, O = Splunk, CN = SplunkCommonCA, emailAddress = support@splunk.com
verify error:num=19:self signed certificate in certificate chain
verify return:0
write W BLOCK
Certificate chain
 0 s:/CN=SplunkServerDefaultCert/O=SplunkUser
   i:/C=US/ST=CA/L=San Francisco/O=Splunk/CN=SplunkCommonCA/emailAddress=support@splunk.com
 1 s:/C=US/ST=CA/L=San Francisco/O=Splunk/CN=SplunkCommonCA/emailAddress=support@splunk.com
   i:/C=US/ST=CA/L=San Francisco/O=Splunk/CN=SplunkCommonCA/emailAddress=support@splunk.com

Yes, the certs are from Splunk.

Thank you

[tkopchak]

Yep, that's the default self-signed cert that comes with Splunk like I suspected.  There's likely no way to fix that on a Cloud trial (and you'll have to disable SSL validation for testing) but you won't have to do that on a production Splunk Cloud stack.