Note about how HEC events are parsed based on the Endpoint configured

[splunkreal]

Hello, a quick note from support to explain how events are handled on HEC (on HF for instance) :

https://help.splunk.com/en/splunk-enterprise/get-started/get-data-in/9.3/get-data-with-http-event-co...

Event endpoint:

1. The event is pre-structured JSON — event boundaries are defined by the JSON structure itself

1. LINE_BREAKER and EVENT_BREAKER settings in props.conf do NOT apply because the JSON payload already defines what constitutes an event

1. However, other props. conf/transforms.conf rules from a TA will still apply if the sourcetype matches — specifically index-time transforms (field extractions, routing, etc.)

1. The sender controls metadata (host, source, sourcetype, index, time) directly in the JSON payload, and individual events in a batch can override token defaults independently

Raw endpoint:

1. Data is treated as raw, unstructured text — just like file-based inputs

1. LINE_BREAKER (regex-based) and EVENT_BREAKER (intelligent breaking) rules from props.conf fully apply for the assigned sourcetype, matching the behaviour of file-monitored inputs

1. The sourcetype is determined by the HEC token configuration, not the payload

1. If a TA installed on the HF has props.conf rules matching that sourcetype (e.g., SHOULD_LINEMERGE, TIME_FORMAT, INDEXED_EXTRACTIONS), those rules will run at index time

1. For batch raw JSON events, props.conf must be configured correctly — e.g., INDEXED_EXTRACTIONS = json for JSON arrays, or SHOULD_LINEMERGE = false for line-delimited events

A TA typically ships two types of config:

props.conf / transforms.conf (index-time rules) — These apply on the HF only if the sourcetype of the incoming HEC data matches stanzas in the TA. With /raw, the full parsing pipeline runs (line breaking, timestamp parsing, field transforms). With /event, only the transforms/field extraction rules apply — not line breaking.

[jasonmiller11tt]

/event uses pre-structured JSON, so line breaking is skipped. /raw follows the normal parsing pipeline, meaning props.conf settings like LINE_BREAKER and INDEXED_EXTRACTIONS apply. The key is ensuring the incoming sourcetype matches your TA configuration. Thanks for the clear explanation!

[PickleRick]

That's also not entirely true. When you say "expects already-defined json events" one could interpret it as "you can only ingest events which are json structures using this endpoint" which is completely untrue. You can ingest any data you want, you just have to use the enpoint by means of sending appropriately formated json event containing the raw event as "event" field. See https://help.splunk.com/en/data-management/get-data-in/get-data-into-splunk-enterprise/10.4/get-data... for details.

[henry_collins]

I think that's a really helpful summary. The biggest takeaway is that /event expects already-defined JSON events, so event breaking settings aren't used, while /raw goes through the normal parsing pipeline where props.conf settings like LINE_BREAKER and SHOULD_LINEMERGE matter. Keeping that distinction in mind makes it much easier to choose the right endpoint for your data.

[PickleRick]

Yup, but the rules regarding HEC are not Kafka-addon-specific 🙂

[PickleRick]

This is not 100% true. Most is but not all.

With /event endpoint:

1. The source _can_ provide index/source/sourcetype/host values but if it doesn't the input assumes the default values. Also if a source requests that event be sent to an index which is not allowed with the provided HEC token, the request will fail
2. Unless explicitly requested with a proper URI parameter, timestamp recognition does _not_ happen with /event endpoint. And even if you want Splunk to parse timestamp but provide one on your own, time parsing won't happen.

For /raw endpoint:

1. To be fully honest, I don't think I've ever tested it with a not-line-oriented sourcetype but I wouldn't be surprised if it did breaking on a per-line basis. But that's just my hunch. Might as well be as you're saying so just test it for yourself if you wanna rely on it. Most notably I'm not sure how the input will react to different parts of a multiline events sent with different HTTP requests. I would be very surprised if it had a way to "correlate" those two separate requests into a single line. So test before use!
2. You _can_ provide destination metadata with your request by means of URI parameters or custom headers. But you have to do it for a whole HTTP request, not for each single event separately.

[splunkreal]

Thanks @PickleRick  for your reply, we use Kafka Connect for your information with event endpoint. The most important is HF addons in most case will apply parsing, fields extractions... however looks like timestamp will be controlled by Kafka.