topic Re: Note about how HEC events are parsed based on the Endpoint configured in Getting Data In

Note about how HEC events are parsed based on the Endpoint configured

splunkreal — Fri, 17 Jul 2026 15:47:50 GMT

Hello, a quick note from support to explain how events are handled on HEC (on HF for instance) :

https://help.splunk.com/en/splunk-enterprise/get-started/get-data-in/9.3/get-data-with-http-event-collector/http-event-collector-rest-api-endpoints

Event endpoint:

The event is pre-structured JSON — event boundaries are defined by the JSON structure itself

LINE_BREAKER and EVENT_BREAKER settings in props.conf do NOT apply because the JSON payload already defines what constitutes an event

However, other props. conf/transforms.conf rules from a TA will still apply if the sourcetype matches — specifically index-time transforms (field extractions, routing, etc.)

The sender controls metadata (host, source, sourcetype, index, time) directly in the JSON payload, and individual events in a batch can override token defaults independently

Raw endpoint:

Data is treated as raw, unstructured text — just like file-based inputs

LINE_BREAKER (regex-based) and EVENT_BREAKER (intelligent breaking) rules from props.conf fully apply for the assigned sourcetype, matching the behaviour of file-monitored inputs

The sourcetype is determined by the HEC token configuration, not the payload

If a TA installed on the HF has props.conf rules matching that sourcetype (e.g., SHOULD_LINEMERGE, TIME_FORMAT, INDEXED_EXTRACTIONS), those rules will run at index time

For batch raw JSON events, props.conf must be configured correctly — e.g., INDEXED_EXTRACTIONS = json for JSON arrays, or SHOULD_LINEMERGE = false for line-delimited events

A TA typically ships two types of config:

props.conf / transforms.conf (index-time rules) — These apply on the HF only if the sourcetype of the incoming HEC data matches stanzas in the TA. With /raw, the full parsing pipeline runs (line breaking, timestamp parsing, field transforms). With /event, only the transforms/field extraction rules apply — not line breaking.

Re: Note about how HEC events are parsed based on the Endpoint configured

PickleRick — Fri, 17 Jul 2026 14:05:44 GMT

This is not 100% true. Most is but not all.

With /event endpoint:

The source _can_ provide index/source/sourcetype/host values but if it doesn't the input assumes the default values. Also if a source requests that event be sent to an index which is not allowed with the provided HEC token, the request will fail
Unless explicitly requested with a proper URI parameter, timestamp recognition does _not_ happen with /event endpoint. And even if you want Splunk to parse timestamp but provide one on your own, time parsing won't happen.

For /raw endpoint:

To be fully honest, I don't think I've ever tested it with a not-line-oriented sourcetype but I wouldn't be surprised if it did breaking on a per-line basis. But that's just my hunch. Might as well be as you're saying so just test it for yourself if you wanna rely on it. Most notably I'm not sure how the input will react to different parts of a multiline events sent with different HTTP requests. I would be very surprised if it had a way to "correlate" those two separate requests into a single line. So test before use!
You _can_ provide destination metadata with your request by means of URI parameters or custom headers. But you have to do it for a whole HTTP request, not for each single event separately.

Re: Note about how HEC events are parsed based on the Endpoint configured

splunkreal — Fri, 17 Jul 2026 15:51:37 GMT

Thanks @PickleRick for your reply, we use Kafka Connect for your information with event endpoint. The most important is HF addons in most case will apply parsing, fields extractions... however looks like timestamp will be controlled by Kafka.

Re: Note about how HEC events are parsed based on the Endpoint configured

PickleRick — Fri, 17 Jul 2026 15:53:22 GMT

Yup, but the rules regarding HEC are not Kafka-addon-specific 🙂

Re: Note about how HEC events are parsed based on the Endpoint configured

henry_collins — Fri, 17 Jul 2026 22:06:53 GMT

I think that's a really helpful summary. The biggest takeaway is that /event expects already-defined JSON events, so event breaking settings aren't used, while /raw goes through the normal parsing pipeline where props.conf settings like LINE_BREAKER and SHOULD_LINEMERGE matter. Keeping that distinction in mind makes it much easier to choose the right endpoint for your data.

Re: Note about how HEC events are parsed based on the Endpoint configured

PickleRick — Sat, 18 Jul 2026 10:58:47 GMT

That's also not entirely true. When you say "expects already-defined json events" one could interpret it as "you can only ingest events which are json structures using this endpoint" which is completely untrue. You can ingest any data you want, you just have to use the enpoint by means of sending appropriately formated json event containing the raw event as "event" field. See https://help.splunk.com/en/data-management/get-data-in/get-data-into-splunk-enterprise/10.4/get-data-with-http-event-collector/format-events-for-http-event-collector for details.

Re: Note about how HEC events are parsed based on the Endpoint configured

jasonmiller11tt — Sun, 19 Jul 2026 06:51:37 GMT

/event uses pre-structured JSON, so line breaking is skipped. /raw follows the normal parsing pipeline, meaning props.conf settings like LINE_BREAKER and INDEXED_EXTRACTIONS apply. The key is ensuring the incoming sourcetype matches your TA configuration. Thanks for the clear explanation!

Re: Note about how HEC events are parsed based on the Endpoint configured

cartergray70543 — Mon, 20 Jul 2026 08:13:17 GMT

Good points. I'd especially recommend testing /raw with multiline events before relying on it, as behavior across separate requests can be tricky. The note about /event only parsing timestamps when explicitly requested is also an important distinction.

Re: Note about how HEC events are parsed based on the Endpoint configured

splunkreal — Mon, 20 Jul 2026 11:47:30 GMT

Hi @PickleRick

We use /event endpoint, our Kafka expert told me he extracts the “message” field and sends it to our HEC through the connector. Does this mean HEC receives text or json events?

He also told me messages are delimited in json, providing source and sourcetype for instance however I understand we don’t control timestamp from Splunk’s side. It’s better than raw endpoint as otherwise he would need to add delimiter on the Vector’s agent source or hardcoded for network logs for example.

Thanks.

Re: Note about how HEC events are parsed based on the Endpoint configured

PickleRick — Mon, 20 Jul 2026 13:28:23 GMT

Well, I can't know how your Kafka pipeline is configured.

But the thing is that when you send events to the HEC endpoint they look like this (quote from the docs; a syntactically erroneous one since json doesn't allow comments 😁)

{
    "time": 1426279439, // epoch time
    "host": "localhost",
    "source": "random-data-generator",
    "sourcetype": "my_sample_data",
    "index": "main",
    "event":  "Hello world!" 
}

All fields except "event" are optional - if there is no time supplied, Splunk will assume current time (or if sent with ?auto_extract_timestamp=true it wil be parsed from the event); for the rest of the fields Splunk will assume the defaults defined for particular HEC token. You can also send additional indexed fields as a "fields" structure.

Judging from your descripiton I'd assume that whatever the "message" field on your Kafka side is, it's getting pushed as "event" field into the HEC input. But what is in your "message" field in Kafka - that I don't know.

And if you use /services/collector/event?auto_extract_timestamp=true and _not_ provide the "time" field, Splunk will parse the timestamp out of the event regarding to the normal settings for the sourcetype. Of course if you have a reliable timestamp at the Kafka level already, you can just as well send it as "time" field and save Splunk some time parsing it again 🙂