Getting Data In

HTTP Event Collector batch post and HTTP errors

yotamcp
Engager

Hi,

I've started using HEC to push data to my Splunk Enterprise instance and noticed the errors I get.

For example, sending this:

 

{"aa": "hello world"}

 

Results in:

 

{
  "text": "No data",
  "code": 5
}

 

 

However, when sending events in batches, I will only get this error if the first event I send is problematic:

 

{"event": "hello world"}
{"aa": "hello world"}

 

Results in:

 

{
  "text": "Success",
  "code": 0
}

 

 

Because I need to know that all my events were sent successfully (and "acks" are not an option, considering I send data to Splunk Cloud as well), is there anything I can do (other than sending each event by itself)?

Labels (1)
Tags (2)
0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @yotamcp 

You must be using /services/collector HEC endpoint. event: <your data> is the format when you send data to collector endpoint and only if it is JSON. In your first example there was no event:<> format hence splunk HEC ignored it in second example you have followed the format.

if you wanted to send raw data like any non JSON use /services/raw/ HEC endpoint. You can send multiple events together in a batch. All combination of examples exist here,

Use cURL to manage HTTP Event Collector tokens, events, and services - Splunk Documentation

https://docs.splunk.com/Documentation/Splunk/8.2.0/Data/HECExamples

---

An upvote would be appreciated and accept solution if it helps!

Tags (4)
0 Karma

yotamcp
Engager

I understand all that.

What I was trying to explain was that in a batch, I can send data like this, and get a "Success" message:

{ ... correct format ... }
{ ... correct format ... }
{ ... correct format ... }
{ ... inccorrect format ... }
{ ... correct format ... }
{ ... correct format ... }

Or I can send data like this and get an error:

{ ... inccorrect format ... }
{ ... correct format ... }
{ ... correct format ... }
{ ... correct format ... }
{ ... correct format ... }
{ ... correct format ... }

 

What I wanted to know, is if there is a way to send batch data, and fail the entire bulk on a single incorrect event (atomically). 

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!