topic Re: Sourcetype for JSON data fails to extract the timestamp in Getting Data In

Sourcetype for JSON data fails to extract the timestamp

mmaaxx — Wed, 09 Jul 2025 18:56:57 GMT

I feed data to Splunk using the HTTP Event Collector, sample event:

{

"event":{

"event_id": "58512040",

"event_name": "Access Granted",

...

"event_local_time_with_offset":"2025-07-09T14:46:28+00:00",

"sourcetype": "BBL_splunk_pacs"

}

I set up datasource type BBL_splunk_pacs (see screenshot below)

When I search for the events, I get:

I see 2 issues:

_time is not parsed correctly from the event_local_time_with_offset.
Most of the time, randomly (?), we get all event fields duplicated, and sometimes they are not duplicated.

Any idea what I may be doing wrong? Thank you.

Re: Sourcetype for JSON data fails to extract the timestamp

mmaaxx — Wed, 09 Jul 2025 18:57:52 GMT

Here is a screenshot of the source type:

Re: Sourcetype for JSON data fails to extract the timestamp

richgalloway — Wed, 09 Jul 2025 19:25:15 GMT

Which HEC endpoint are you sending to? The behavior is different depending on the endpoint. The /event endpoint will ignore props settings, but the /raw endpoint honors them.

The TIME_FORMAT value doesn't match the data. Try using %Y-%m-%dT%H:%M:%S%:z

Re: Sourcetype for JSON data fails to extract the timestamp

mmaaxx — Wed, 09 Jul 2025 19:58:17 GMT

Thank you for the suggestion!

I tried "%Y-%m-%dT%H:%M:%S%:z" - same results (seems like timestamp extraction is ignored 😞 ).
I also validated my time format in PHP and Python strptime("2025-07-09T15:50:20+00:00", "%Y-%m-%dT%H:%M:%S%z") - it seems to work.

Yes, I do send to /event.

When I tried sending to /raw I get this (seems like it considers the RAW HTTP request data as "data"):

It doesn't seem to be related to parsing.

Re: Sourcetype for JSON data fails to extract the timestamp

richgalloway — Wed, 09 Jul 2025 20:14:52 GMT

Sending to the /event endpoint skips the props settings. Splunk expects the metadata to be included in the HEC packet. See https://docs.splunk.com/Documentation/Splunk/9.4.2/Data/FormateventsforHTTPEventCollector#Event_metadata for the supported metadata fields.

Consider adding auto_extract_timestamp=true to the HEC URL to tell Splunk to do timestamp parsing. See https://splunk.my.site.com/customer/s/article/Timestamp-Not-Extracted-from-JSON-Payload-When-Using-HEC-event-Endpoint-Without-auto-extract-timestamp-true

Re: Sourcetype for JSON data fails to extract the timestamp

mmaaxx — Wed, 09 Jul 2025 20:51:22 GMT

Thank you for clarifying how it works!

Sending "time" along with the "event" - fixed the timestamp issue, and setting Indexed Extraction to none - fixed the duplicated fields, as all fields are essentially parsed in the application that feeds the data to the /event endpoint.

Thank you!