Source type Timestamp settings.

[spilepich]

Hi,

I'm trying to set up a source type that parses the date from an inner field (message.date in the below example) however the _time value is always set as the processed time. Any guesses as to what i might be doing wrong?

The configuration is as follows:

This is a sample event.

{"message":{"_id":"some_value","date":"2018-07-18T04:40:58.071Z","type":"fsa","description":"Login required","client_id":"some_value","client_name":"some_value","ip":"some_value","user_agent":"Chrome 67.0.3396 / Windows 10 0.0.0","details":{"body":{"tenant":"some_value"},"qs":{"client_id":"some_value","response_type":"id_token","response_mode":"web_message","redirect_uri":"some_value","scope":"openid email profile","audience":"some_value","leeway":"60","state":"some_value","nonce":"some_value","prompt":"none","auth0Client":"some_value","tenant":"hirer"},"connection":null,"error":{"message":"Login required","oauthError":"login_required","type":"oauth-authorization"}},"hostname":"a","session_connection":null,"session_connection_id":null,"audience":"o","scope":["openid","email","profile"],"isMobile":false},"severity":"info"}


[eavent_splunk]

This was a HEC input going to the standard endpoint, which expects data to be formatted a certain way:
http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/FormateventsforHTTPEventCollector#Event_metad...

It will not extract timestamps from JSON fields other than "time" in epoch format.
To resolve, use the raw endpoint, which will do default field extraction as per sourcetype configuration:
http://docs.splunk.com/Documentation/Splunk/7.1.2/Data/FormateventsforHTTPEventCollector#Raw_event_p...

Hope that helps!

[CarsonZa]

i would try what @poete is suggesting instead of changing it to _json leave it as default and then add your timestamp format in. I tested it with your sample event and it works great.

[poete]

Hello @spilepich,

why bother with configuring the extraction? I did import the sample of data successfuly using the Auto Extraction.

But maybe you are trying to do something I did not get. Did, you try with Auto?

[spilepich]

yes, that's the first thing i tried. The data imports with the load time as _time instead of what is in message.date

[poete]

Hello @spilepich,
Well, I think you get both. In _time, you get the value extracted from message.date, which enable to display it according to the user timezone, as configured in Splunk. In message.date, you get the value contained in the original json event.

This is the standard behaviour in Splunk