Splunk Dev

Splunk using HTTP Event Collector (HEC) with services/collector/raw endpoint to send whole datafile

RetailOperation
Explorer

Hello,

we want to send a whole file via the http event collector to the endpoint services/collector/raw as descirbed here:
http://dev.splunk.com/view/event-collector/SP-CAAAE8Y

"curl https://http-inputs-.splunkcloud.com/services/collector/raw -H "X-Splunk-Request-Channel: FE0ECFAD-13D5-401B-847D-77833BD77131" -H "Authorization: Splunk BD274822-96AA-4DA6-90EC-18940FB2414C" -d '' -v"

As we are on Splunk 7 we do not use the "X-Splunk-Request-Channel"

The data is recieved by an heavy forwarder and parsed to the indexer cluster.
The data is arriving at the assigned index but the problem is that splunk handles the whole file as one event.
I created a sourcetype on a test environment which breaks up the events in single events when i upload the file manually.
But when i send this file to our productive environment via the HEC with /raw endpoint the file is still indexed as one event.

I tried to configure the sourcetype on all indexers in the index cluster -> was nor working
I tried to configure the sourcetype also at the heavy forwarders -> not working

Any suggestions what to do here?

thanks in advance

Tags (1)
0 Karma

starcher
Influencer

The event endpoint expects properly formed JSON and things broken into events. It bypasses parsing queues. If you send to RAW you treat it like any file pickup ingestion and do your own line breaking etc as you would any text stream multi line event.

0 Karma
Get Updates on the Splunk Community!

Unlock Database Monitoring with Splunk Observability Cloud

  In today’s fast-paced digital landscape, even minor database slowdowns can disrupt user experiences and ...

Purpose in Action: How Splunk Is Helping Power an Inclusive Future for All

At Cisco, purpose isn’t a tagline—it’s a commitment. Cisco’s FY25 Purpose Report outlines how the company is ...

[Upcoming Webinar] Demo Day: Transforming IT Operations with Splunk

Join us for a live Demo Day at the Cisco Store on January 21st 10:00am - 11:00am PST In the fast-paced world ...