HEC token use for the source to Splunk

uagraw01 — Wed, 25 Sep 2024 14:13:18 GMT

Hello Splunkers!!

I have ingested data into Splunk from the source system using the URI "https://localhost:8088/services/collector" along with the HEC token. However, the data is not being displayed in Splunk with the appropriate sourcetype parsing, which is affecting the timestamp settings for the events.

The sourcetype and timestamp are currently being displayed as below.

My actual props.conf setting as below :

[agv_voot] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true category = Custom KV_MODE = json pulldown_type = 1 TIME_PREFIX = ^\@timestamp TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N TIMESTAMP_FIELDS = @timestamp TRANSFORMS-trim_timestamp = trim_long_timestamp

transforms.conf [trim_long_timestamp] REGEX = (\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3})\d+(-\d{2}:\d{2}) FORMAT = $1

Please help to fix the proper parsing with correct sourcetype and timestamp.

Re: HEC token use for the source to Splunk

PickleRick — Wed, 25 Sep 2024 10:55:38 GMT

https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fevent

services/collector/event

Sends timestamped events to HTTP Event Collector using the Splunk platform JSON event protocol when auto_extract_timestamp is set to "true" in the /event URL.

An example of a timestamp is: 2017-01-02 00:00:00.
If there is a timestamp in the event's JSON envelope, Splunk honors that timestamp first.
If there is no timestamp in the event's JSON envelope, the merging pipeline extracts the timestamp from the event.
If "time=xxx" is used in the /event URL then auto_extract_timestamp is disabled.
Splunk supports timestamps using the Epoch format.

In other words - unless you specify your URI as /services/collector/event?auto_extract_timestamp=true your timestamp will _not_ be extracted from the event itself (Splunk will not even bother looking for it - it will either get the data from the json envelope or will assume current timestamp if there is no timestamp in the envelope). And even if the auto_extract_timestamp parameter is set to true, in cases listed above extraction is not performed either.

topic Re: HEC token use for the source to Splunk in Splunk Enterprise

HEC token use for the source to Splunk

Re: HEC token use for the source to Splunk

services/collector/event