Splunk Dev

Splunk using HTTP Event Collector (HEC) with services/collector/raw endpoint to send whole datafile

RetailOperation
Explorer

Hello,

we want to send a whole file via the http event collector to the endpoint services/collector/raw as descirbed here:
http://dev.splunk.com/view/event-collector/SP-CAAAE8Y

"curl https://http-inputs-.splunkcloud.com/services/collector/raw -H "X-Splunk-Request-Channel: FE0ECFAD-13D5-401B-847D-77833BD77131" -H "Authorization: Splunk BD274822-96AA-4DA6-90EC-18940FB2414C" -d '' -v"

As we are on Splunk 7 we do not use the "X-Splunk-Request-Channel"

The data is recieved by an heavy forwarder and parsed to the indexer cluster.
The data is arriving at the assigned index but the problem is that splunk handles the whole file as one event.
I created a sourcetype on a test environment which breaks up the events in single events when i upload the file manually.
But when i send this file to our productive environment via the HEC with /raw endpoint the file is still indexed as one event.

I tried to configure the sourcetype on all indexers in the index cluster -> was nor working
I tried to configure the sourcetype also at the heavy forwarders -> not working

Any suggestions what to do here?

thanks in advance

Tags (1)
0 Karma

starcher
Influencer

The event endpoint expects properly formed JSON and things broken into events. It bypasses parsing queues. If you send to RAW you treat it like any file pickup ingestion and do your own line breaking etc as you would any text stream multi line event.

0 Karma
Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Your Voice Matters! Help Us Shape the New Splunk Lantern Experience

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Building Momentum: Splunk Developer Program at .conf25

At Splunk, developers are at the heart of innovation. That’s why this year at .conf25, we officially launched ...