topic Re: timestamp contain T between date and time in Getting Data In

timestamp contain T between date and time

jayannah — Mon, 16 Jun 2014 14:26:49 GMT

Hi All

The timestamp is in the format T , e.g: 2014-06-05T05:03:53-07:00

Is there any configuration supported in SPlunk to automatically recognize and extract this timestamp to _time ?

Re: timestamp contain T between date and time

somesoni2 — Mon, 16 Jun 2014 14:42:57 GMT

Add following in your sourcetype configuration in props.conf.

[YourSourceType]
..Other stuffs..
...
TIME_FORMAT = %Y-%m-%dT%H:%M:%S%z

Re: timestamp contain T between date and time

jayannah — Mon, 16 Jun 2014 20:18:42 GMT

Let me be more specific about my question.

Note that the "T" appears literally in the string, to indicate the beginning of the time element, as specified in ISO 8601.

So, I'm of looking if there is way to mention about the timeformat used is ISO-8601. Like we mention TZ for timezone. Like this, is there something similar to mention say TF=ISO-8601 to indicate the timeformat used in the log is ISO-8601 compliance.

Hope this clarifies my question.

Re: timestamp contain T between date and time

somesoni2 — Mon, 16 Jun 2014 20:28:58 GMT

I created some sample log file with timestamp format specified by you and bunch of other format in same log file, and splunk is able to recognize it(without specifying Time format). Do you face any issue with your logs?
Sample test log created.

2014-06-05T05:03:53-07:00 name=this
2014-06-05 05:04:53-07:00 name=is
06/05/2014 05:05:53 -07:00 name=datetime
06/05/2014:05:06:53 -07:00 name=test

Re: timestamp contain T between date and time

jayannah — Tue, 17 Jun 2014 05:39:48 GMT

The _time is getting the correct value, but if I see it in data previews screen, it is selecting correct time from incorrect data 🙂

E.g:Sample log edited from actual.
2014-06-05T05:03:53-07:00 server8 monitor: "10.29.224.229" - [05/Jun/2014:05:03:53 -0700] "GET /login?userId=10099234081 HTTP/1.1" 200 87323"-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)" 0.543 "-/-" www.mywebsite.com

Here, in the preview screen, the _time is extracted from "05T05:03:53-07:00 server8 monitor: "10.29.224.229" - [05/Jun/2014" as highlighted in the preview Window.

Re: timestamp contain T between date and time

grijhwani — Tue, 17 Jun 2014 11:02:34 GMT

If you have multiple sources for the timestamp and you want a specific field to be use, then you will need to be explicit in the source configuration about which field(s) to use (props.conf?). It is not the format that matters, it is the "where". You can't rely on defaults. It all depends on how the analysis tree is implemented within Splunk, otherwise, and that you have no control over.