topic Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING in Getting Data In

Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Tue, 01 Dec 2015 20:16:22 GMT

I have the 15 day trial version of Splunk Cloud. The Http Event Collector documentation http://dev.splunk.com/view/event-collector/SP-CAAAE7F says: Note: To turn on HTTP Event Collector in Splunk Cloud, file a request ticket with Splunk Support.

Since this is a trial version I am not allowed to submit a support ticket. How do I get Http Event Collector enabled?

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Tue, 01 Dec 2015 21:12:43 GMT

Hi @simpkins1958.

You can now enable HTTP Event Collector yourself in Trial / Single Instance. Go to Settings->Data Inputs->HTTP Event Collector from there you can enable the collector and create a token.

In order to create requests, you need to prefix the URI of your cloud instance with "input-", i.e. see the curl below

curl -k https://input-prd-p-j65vnzzl9wc8.cloud.splunk.com:8088/services/collector -H 'Authorization: Splunk  498FEC9B-86E2-4CD0-B489-4A55E2D52B07' -d '{"event":"event1"} {"event":"event2"}'

Notice I've added "input-" then the instance. Also the port + /services/collector endpoint are there.

As a side note, Splunk Cloud trial uses a self-signed cert, so you need to disabled cert validation if using HTTPS which is what the '-k' switch does with curl.

Let me know if you have any issues.

Thanks
Glenn

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Tue, 01 Dec 2015 22:42:04 GMT

Thanks Glenn. I added the prefix and postfix and I am able to get it to work with curl. But it is not working when using from our C# code using our version of Splunk.Logging.Common. HTTPS is working fine from our C# code when sending data to an on prem instance of Splunk, but not Splunk Cloud. I will keep investigating.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Tue, 01 Dec 2015 22:46:21 GMT

And we have disabled cert validation using:

            ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) =>
            {
                return true;
            };

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Tue, 01 Dec 2015 22:50:41 GMT

InnerException is:

InnerException {"The request was aborted: Could not create SSL/TLS secure channel."} System.Exception {System.Net.WebException}

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Tue, 01 Dec 2015 22:55:23 GMT

Try setting the security policy to use TLS.

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Tue, 01 Dec 2015 23:09:26 GMT

Tried setting the security policy to use TLS still not working with Splunk Cloud.

Tried again with my local Splunk server using HTTPS and working fine.

I set a break point in ServerCertificateValidationCallback, which is never getting hit when trying to send to Splunk Cloud but is getting hit when sending to local Splunk Server.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Tue, 01 Dec 2015 23:19:37 GMT

Based on some googling around I also tried:

            ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;

Also not working.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Tue, 01 Dec 2015 23:24:39 GMT

It sounds like this is a .NET related issue based on the fact that you can successfully curl. Can you try using HttpClient directly and see if you are able to send data?

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Tue, 01 Dec 2015 23:25:33 GMT

I am going to do a test against a trial instance we just deployed and see if that works.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Wed, 02 Dec 2015 00:56:59 GMT

Jim, what version of Windows are you running? Splunk Cloud uses Elliptical Curve certs. I've noticed on Stack Overflow, some folks having issues with this type of cert.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Wed, 02 Dec 2015 02:11:16 GMT

My dev VM I am running from is Windows 2008 R2. I will try on a Windows 2012 server.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Wed, 02 Dec 2015 02:35:53 GMT

OK, I found something based on this article.

Try this:

ServicePointManager.Expect100Continue = true;

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Wed, 02 Dec 2015 02:38:10 GMT

Looks like there are multiple reasons this can happen. I am going to try Windows 10 and see if I experience the same.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Wed, 02 Dec 2015 03:05:45 GMT

I tried on Windows 2012 R2 server, and it is also not working with splunk cloud but works with my local splunk server.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Wed, 02 Dec 2015 03:16:28 GMT

Yeah, if you look in the Stack Overflow, people were hitting it with 2012. The difference is that you are not running with an Elliptical Curve cert locally, and that I am guessing is where the issue lies.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Wed, 02 Dec 2015 03:47:12 GMT

I tried on Windows 10, and am hitting the same issue. My strong hunch at this point is the Elliptical Curve certs may be the issue, though I cannot say for sure.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Thu, 03 Dec 2015 23:39:48 GMT

Want to give an update here. I am working directly with Microsoft now as it looks like the "may" be a bug in the HTTP stack for .NET that is preventing allowing you to ignore self-signed Elliptical Curved Certs (which Splunk Cloud uses). I'll update back when I know for sure and if there any next steps.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

simpkins1958 — Thu, 03 Dec 2015 23:43:40 GMT

Thanks for the update.

Re: Splunk Cloud Trial and Http Event Collector - NOT WORKING

gblock_splunk — Thu, 10 Dec 2015 01:52:50 GMT

New update.

This will not work today in cloud trial or single instance. We are addressing this though and should have this fixed in a few weeks

I have validated that it works fine in a clustered Splunk Cloud environment.

Thanks for your patience.