Getting Data In

Source type Timestamp settings.

New Member

Hi,

I'm trying to set up a source type that parses the date from an inner field (message.date in the below example) however the _time value is always set as the processed time. Any guesses as to what i might be doing wrong?

The configuration is as follows:
alt text

This is a sample event.

{"message":{"_id":"some_value","date":"2018-07-18T04:40:58.071Z","type":"fsa","description":"Login required","client_id":"some_value","client_name":"some_value","ip":"some_value","user_agent":"Chrome 67.0.3396 / Windows 10 0.0.0","details":{"body":{"tenant":"some_value"},"qs":{"client_id":"some_value","response_type":"id_token","response_mode":"web_message","redirect_uri":"some_value","scope":"openid email profile","audience":"some_value","leeway":"60","state":"some_value","nonce":"some_value","prompt":"none","auth0Client":"some_value","tenant":"hirer"},"connection":null,"error":{"message":"Login required","oauthError":"login_required","type":"oauth-authorization"}},"hostname":"a","session_connection":null,"session_connection_id":null,"audience":"o","scope":["openid","email","profile"],"isMobile":false},"severity":"info"}

0 Karma

Splunk Employee
Splunk Employee

This was a HEC input going to the standard endpoint, which expects data to be formatted a certain way:
http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/FormateventsforHTTPEventCollector#Event_metad...

It will not extract timestamps from JSON fields other than "time" in epoch format.
To resolve, use the raw endpoint, which will do default field extraction as per sourcetype configuration:
http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/FormateventsforHTTPEventCollector#Raw_event_p...

Hope that helps!

0 Karma

Contributor

i would try what @poete is suggesting instead of changing it to _json leave it as default and then add your timestamp format in. I tested it with your sample event and it works great.

0 Karma

Builder

Hello @spilepich,

why bother with configuring the extraction? I did import the sample of data successfuly using the Auto Extraction.

But maybe you are trying to do something I did not get. Did, you try with Auto?

0 Karma

New Member

yes, that's the first thing i tried. The data imports with the load time as _time instead of what is in message.date

0 Karma

Builder

Hello @spilepich,
Well, I think you get both. In _time, you get the value extracted from message.date, which enable to display it according to the user timezone, as configured in Splunk. In message.date, you get the value contained in the original json event.

This is the standard behaviour in Splunk