HTTP Event collector won't work - only http 404 when trying to post

[rune_hellem]

Splunk Enterprise - Windows - 8.0.5

I have tried to enable the HTTP Event Collector following this guideline https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/UsetheHTTPEventCollector - made sure that HEC is enabled and then created a token. 

[http://MyScript]
disabled = 0
index = operations
indexes = operations
token = b68999b2-9f22-4b53-ba6e-0a8cfd505251
useACK = 0
description = HTTP EVent collector for script

From file 

D:\Splunk\etc\apps\search\local\inputs.conf

Server restarted - but still 

curl -k "https://splunkindex:8088/services/collector"
{"text":"The requested URL was not found on this server.","code":404}

So whatever I do trying to post an event - fails 

curl -k "https://splunkindex:8088/services/collector" -H "Authorization: Splunk b68999b2-9f22-4b53-ba6e-0a8cfd505251" -d '{"event": "Hello, world!",  "sourcetype": "manual"}'
{"text":"Invalid data format","code":6,"invalid-event-number":0}curl: (3) URL using bad/illegal format or missing URL
curl: (6) Could not resolve host: sourcetype
curl: (3) unmatched close brace/bracket in URL position 7:
manual}'

but at least something is working 

curl -k "https://splunkindex:8088/services/collector/health"
{"text":"HEC is healthy","code":17}

Did also try 

|rest /services/collector/health

but that fails - so I have not fully understod the "| rest" command. 

Finally, ref https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/HTTPEventCollectortokenmanagement it says I can list the existing tokens using command 

curl -k -u admin:password https://splunkindex:8089/servicesNS/admin/splunk_httpinput/data/inputs/http

But I cannot see any refrences to my token in the output.

<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
  <title>http</title>
  <id>https://splunkindex:8089/servicesNS/admin/splunk_httpinput/data/inputs/http</id>
  <updated>2020-08-26T21:40:06+02:00</updated>
  <generator build="a1a6394cc5ae" version="8.0.5"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/servicesNS/admin/splunk_httpinput/data/inputs/http/_new" rel="create"/>
  <link href="/servicesNS/admin/splunk_httpinput/data/inputs/http/_reload" rel="_reload"/>
  <link href="/servicesNS/admin/splunk_httpinput/data/inputs/http/_acl" rel="_acl"/>
  <opensearch:totalResults>0</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>
</feed>

[isoutamo]

Hi

can you post the output of 

splunk btool inputs list http

r. Ismo 

[rune_hellem]

D:\Splunk\bin>splunk btool inputs list http
[http]
_rcvbuf = 1572864
ackIdleCleanup = true
allowSslCompression = true
allowSslRenegotiation = true
dedicatedIoThreads = 2
disabled = 0
enableSSL = 1
evt_dc_name =
evt_dns_name =
evt_resolve_ad_obj = 0
host = SplunkIndex
index = operations
maxSockets = 0
maxThreads = 0
port = 8088
sslVersions = *,-ssl2
useDeploymentServer = 0
[http://MyScript]
_rcvbuf = 1572864
description = HTTP EVent collector for script
disabled = 0
evt_dc_name =
evt_dns_name =
evt_resolve_ad_obj = 0
host = SplunkIndex
index = operations
indexes = operations
token = b68999b2-9f22-4b53-ba6e-0a8cfd505251
useACK = 0

[isoutamo]

Please try to whole url like 

curl -k  https://hec.splunk.com:8088/services/collector/event -H "Authorization: Splunk B5A89CAD-D842-46CC-80A1-017F83D7CFB2" -d '{"event": "hello world"}'

it should contains  /event also.

r. Ismo

[rune_hellem]

Did not help to add /event. If I remove -d it returns the 404. Not really sure what is really wrong since I get three errors. Did try to switch the quotes (single vs double), but that did not help either - so not really sure if the brackets are wrong or if that is just a false positive. 

curl -k "https://splunkindex:8088/services/collector" -H "Authorization: Splunk b68999b2-9f22-4b53-ba6e-0a8cfd505251" -d '{"event": "Hello, world!",  "sourcetype": "manual"}'
{"text":"Invalid data format","code":6,"invalid-event-number":0}curl: (3) URL using bad/illegal format or missing URL
curl: (6) Could not resolve host: sourcetype
curl: (3) unmatched close brace/bracket in URL position 7:
manual}'

[isoutamo]

Please try to remove “ from https... URL should be without those. 

[isoutamo]

Hi

I just enabled HEC and created new token based on that instructions on my workstation with Splunk 8.0.5. Works w/o issues.

(0) $ curl -k https://localhost:8088/services/collector/event -H "Authorization: Splunk 9dd3e284-1619-4070-be37-59250cd52876" -d '{"event": "Hello, world!",  "sourcetype": "manual"}'
{"text":"Success","code":0}
(0) $ curl -k "https://localhost:8088/services/collector/event" -H "Authorization: Splunk 9dd3e284-1619-4070-be37-59250cd52876" -d '{"event": "Hello, world!",  "sourcetype": "manual"}'
{"text":"Success","code":0}

At least on macOS both version "https" and https seems to works w/o issues. 

r. Ismo

[rune_hellem · Accepted solution]

Ahh, finally got it 

curl -k https://splunkindex:8088/services/collector/event -H "Authorization: Splunk b68999b2-9f22-4b53-ba6e-0a8cfd505251" -d "{\"event\": \"Hello, world!\",  \"sourcetype\": \"manual\"}"

This post to the rescue https://community.splunk.com/t5/Getting-Data-In/HTTP-event-collector-error-with-data-format/td-p/352...

Had to use double quotes around -d and escape them inside the brackets.  

And thanks @isoutamo  for helping me!

[Borobest]

Thank you for submitting this solution - after a number of hours of searching for the correct format this works for me. 

[isoutamo]

Or 

-d '{"event": "Hello, world!",  "sourcetype": "manual"}'

Single quotes outside and double inside. Just like we have on those examples.

Or is it so that in windows those haven't any difference?

But good to know that you mange it and it's working now.

r. Ismo 

[rune_hellem]

For some reason, it did not accept single qoutes on my laptop (windows 10)...