topic HTTP Event Collector batch post and HTTP errors in Getting Data In

HTTP Event Collector batch post and HTTP errors

yotamcp — Wed, 23 Jun 2021 08:24:33 GMT

Hi,

I've started using HEC to push data to my Splunk Enterprise instance and noticed the errors I get.

For example, sending this:

{"aa": "hello world"}

Results in:

{ "text": "No data", "code": 5 }

However, when sending events in batches, I will only get this error if the first event I send is problematic:

{"event": "hello world"} {"aa": "hello world"}

Results in:

{ "text": "Success", "code": 0 }

Because I need to know that all my events were sent successfully (and "acks" are not an option, considering I send data to Splunk Cloud as well), is there anything I can do (other than sending each event by itself)?

Re: HTTP Event Collector batch post and HTTP errors

venkatasri — Thu, 24 Jun 2021 07:03:06 GMT

Hi @yotamcp

You must be using /services/collector HEC endpoint. event: <your data> is the format when you send data to collector endpoint and only if it is JSON. In your first example there was no event:<> format hence splunk HEC ignored it in second example you have followed the format.

if you wanted to send raw data like any non JSON use /services/raw/ HEC endpoint. You can send multiple events together in a batch. All combination of examples exist here,

Use cURL to manage HTTP Event Collector tokens, events, and services - Splunk Documentation

https://docs.splunk.com/Documentation/Splunk/8.2.0/Data/HECExamples

---

An upvote would be appreciated and accept solution if it helps!

Re: HTTP Event Collector batch post and HTTP errors

yotamcp — Sun, 27 Jun 2021 06:52:27 GMT

I understand all that.

What I was trying to explain was that in a batch, I can send data like this, and get a "Success" message:

{ ... correct format ... } { ... correct format ... } { ... correct format ... } { ... inccorrect format ... } { ... correct format ... } { ... correct format ... }

Or I can send data like this and get an error:

{ ... inccorrect format ... } { ... correct format ... } { ... correct format ... } { ... correct format ... } { ... correct format ... } { ... correct format ... }

What I wanted to know, is if there is a way to send batch data, and fail the entire bulk on a single incorrect event (atomically).