Getting Data In

Sourcetype for JSON data fails to extract the timestamp

mmaaxx
Explorer

I feed data to Splunk using the HTTP Event Collector, sample event:

{

"event":{

"event_id": "58512040",

"event_name": "Access Granted",

...

"event_local_time_with_offset":"2025-07-09T14:46:28+00:00",

},

"sourcetype": "BBL_splunk_pacs"

}  

 

I set up datasource type BBL_splunk_pacs (see screenshot below)

When I search for the events, I get:

I see 2 issues:

  1. _time is not parsed correctly from the event_local_time_with_offset. 

  2. Most of the time, randomly (?), we get all event fields duplicated, and sometimes they are not duplicated.

     

    Any idea what I may be doing wrong?  Thank you.

     

    mmaaxx_1-1752087006302.png

 

 

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Sending to the /event endpoint skips the props settings.  Splunk expects the metadata to be included in the HEC packet.  See https://docs.splunk.com/Documentation/Splunk/9.4.2/Data/FormateventsforHTTPEventCollector#Event_meta... for the supported metadata fields.

Consider adding auto_extract_timestamp=true to the HEC URL to tell Splunk to do timestamp parsing.  See https://splunk.my.site.com/customer/s/article/Timestamp-Not-Extracted-from-JSON-Payload-When-Using-H...

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Which HEC endpoint are you sending to?  The behavior is different depending on the endpoint.  The /event endpoint will ignore props settings, but the /raw endpoint honors them.

The TIME_FORMAT value doesn't match the data.  Try using %Y-%m-%dT%H:%M:%S%:z

---
If this reply helps you, Karma would be appreciated.

mmaaxx
Explorer

Thank you for the suggestion!

I tried "%Y-%m-%dT%H:%M:%S%:z" - same results (seems like timestamp extraction is ignored 😞  ). 
I also validated my time format in PHP and Python strptime("2025-07-09T15:50:20+00:00", "%Y-%m-%dT%H:%M:%S%z") - it seems to work. 

 

Yes, I do send to /event. 

When I tried sending to /raw I get this (seems like it considers the RAW HTTP request data as "data"):

mmaaxx_0-1752090944427.png

It doesn't seem to be related to parsing. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Sending to the /event endpoint skips the props settings.  Splunk expects the metadata to be included in the HEC packet.  See https://docs.splunk.com/Documentation/Splunk/9.4.2/Data/FormateventsforHTTPEventCollector#Event_meta... for the supported metadata fields.

Consider adding auto_extract_timestamp=true to the HEC URL to tell Splunk to do timestamp parsing.  See https://splunk.my.site.com/customer/s/article/Timestamp-Not-Extracted-from-JSON-Payload-When-Using-H...

---
If this reply helps you, Karma would be appreciated.

mmaaxx
Explorer

Thank you for clarifying how it works! 

Sending "time" along with the "event" - fixed the timestamp issue, and setting Indexed Extraction to none - fixed the duplicated fields, as all fields are essentially parsed in the application that feeds the data to the /event endpoint.

Thank you!

mmaaxx
Explorer

Here is a screenshot of the source type:

mmaaxx_0-1752087465960.png

 

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...