Getting Data In

HTTP event collector -- error with data format?

tamduong16
Contributor

I want to try to inputting a simple event to HTTP event collector just to test if it works. I think it was able to find the web address and also authenticate it with the token value. But I get an error with the invalid data format. What can I do to fix it?
I have the following command:
curl -k -H "Authorization: Splunk B86C5445-76D4-4FAF-A0FA-D8FE2FA49F79" https://localhost:8088/services/collector/event -d '{"event":"testing"}'
With the following result:
{"text":"Invalid data format","code":6,"invalid-event-number":0}

0 Karma

anupagazi
New Member

Just replace event by raw like this:
curl -k https://localhost:8088/services/collector/raw -H "Authorization: Splunk 65652e8c-443d-42b6-9b75-02657b215665" -d '{"event":"This is test http event collector"}'
You can find the answer from this link:
https://medium.com/adarma-tech-blog/splunk-http-event-collectors-explained-2c22e87ab8d2,Just replace event by raw like this:
curl -k https://localhost:8088/services/collector/raw -H "Authorization: Splunk 65652e8c-443d-42b6-9b75-02657b215665" -d '{"event":"This is test http event collector"}'
I found this answer from following link:
https://medium.com/adarma-tech-blog/splunk-http-event-collectors-explained-2c22e87ab8d2

0 Karma

damiensurat
Contributor

I know this is an old post, but it seems that there isn't a good answer to this question, so I felt I should share after recently experiencing the same error.
The error is in my experience is indicative of a formatting issue or expected key / value pairs that are missing in or with your data. I would inspect the data for unexpected characters, white space's, etc as well. An example: I ran into this issue, and when inspecting the format of the data there were white spaces between colons that separate k/v pairs:

Example of data where I saw this error. Although not apparent, there are white spaces between the key: value. When I removed the white spaces, I was able to successfully send the data:
(NOTE: This is metric data going to a metric index).

"{
"source": "t_e_api",
"host": "server1,
"event": "metric",
"fields":
{
"ftpTimeLimit": "1",
"group": "",
"metric_name": "c_u_used",
"agents": "",
"accountGroupName": "Is1",
"url": "",
"type": "1",
"interval":"4",
"enabled":"1",
"testName": "test1",
"_value": 3022222,
"sipTimeLimit": "1",
"httpTimeLimit": "3",
"aid": 170000,
"pageLoadTimeLimit": "7",
"test_id": 110101,
"testType": "Web - HTTP Server"
},
"time": 1571420739
}"

This adjustment (removal of white spaces) fixed the issue:
"{
"source":"t_e_api",
"host":"server1,
"event":"metric",
"fields":
{
"ftpTimeLimit":"1",
"group":"",
"metric_name":"c_u_used",
"agents":"",
"accountGroupName":"Is1",
"url":"",
"type":"1",
"interval":"4",
"enabled":"1",
"testName":"test1",
"_value":3022222,
"sipTimeLimit":"1",
"httpTimeLimit":"3",
"aid":170000,
"pageLoadTimeLimit":"7",
"test_id":110101,
"testType":"Web - HTTP Server"
},
"time":1571420739
}"

0 Karma

chli_splunk
Splunk Employee
Splunk Employee

Need to add sourcetype in your event like {"event":"testing", "sourcetype": "st"}

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Why the duplicate post? You already had a thread going... Anyways, try escaping the double quotes (curl may not like it the way it is):

curl -k -H "Authorization: Splunk B86C5445-76D4-4FAF-A0FA-D8FE2FA49F79" https://localhost:8088/services/collector/event -d '{\"event\":\"testing\"}'
0 Karma

tamduong16
Contributor

I still have the same error 😞

0 Karma

ddrillic
Ultra Champion

For the record, this one worked for us -

curl "https://<HEC service>:443/services/collector/event" -H "Authorization: Splunk f5b9eac2-7319-4dfb-80d0-86f44a9785cd" -k -d "{\"host\":\"xxxx\",\"sourcetype\":\"test_hec\",\"source\":\"test\",\"event\":{\"message\":\"ERROR\",\"code\":\"401\"}}"
0 Karma
Get Updates on the Splunk Community!

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...