topic Re: Time zone in TIME_FORMAT in Getting Data In

Time zone in TIME_FORMAT

vbumgarner — Wed, 02 Mar 2011 02:51:26 GMT

Given this timestamp:

01/Mar/2011:17:25:49.666+0000

What is the right format?
I'm leaning towards:

TIME_FORMAT = %d/%b/%Y:$H:%M:%S.%3N+%z

but I'm not quite sure what this means from the extended strptime documentation:

%z, %::z, %:::z GNU libc support.

Re: Time zone in TIME_FORMAT

Stephen_Sorkin — Wed, 02 Mar 2011 02:58:34 GMT

Unfortunately we don't ship our "DateTimeTest" utility that allows easy testing of strptime/strftime functionality, please file an ER for that to be included from the CLI/UI. Here's my findings (first note that you have $H rather than %H):

[ssorkin@MrT current]$ src/util/tests/DateTimeTest '%d/%b/%Y:%H:%M:%S.%3N+%z' '01/Mar/2011:17:25:49.666+0000'
01/Mar/2011:09:25:49.666+-0800
[ssorkin@MrT current]$ src/util/tests/DateTimeTest '%d/%b/%Y:%H:%M:%S.%3N%z' '01/Mar/2011:17:25:49.666+0000'
01/Mar/2011:09:25:49.666-0800

So basically, %z does essentially the right thing, but the + is unnecessary and actually causes wrong results.

As a clarification, the : in %z says how to split the time zone offset. One : looks like -08:00 for PST, two : looks like -08:00:00.

Re: Time zone in TIME_FORMAT

vbumgarner — Wed, 02 Mar 2011 03:28:19 GMT

Perfect. Thanks.

Re: Time zone in TIME_FORMAT

emma — Wed, 25 Apr 2012 19:39:58 GMT

Actually, you should use the new preview feature (shipped in 4.3) to test any tz configs.

Re: Time zone in TIME_FORMAT

peterzhoupeterz — Tue, 21 Aug 2012 19:35:19 GMT

%z is -0400 This format is not standard. if your machine is configure as Eastern Date Time

%Z is EDT if your machine is configure as Eastern Date Time, not too much use for storing it in data base. By the way I live in New York.

%:z is -04:00 That is the one most useful in hours and minutes. It can be used across computer languages

%::z is -04:00:00 It is over kill. we don't need second for time zone

%:::z is even more over kill, no use in reality

Re: Time zone in TIME_FORMAT

bwooden — Tue, 09 Jul 2013 21:22:35 GMT

Actually, that's true. Though actually, it was posted 2 years later.

Preview is great for manipulation & validation of timestamp extraction before implementation.

Also: http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition#Enhanced_strptime.28.29_support

Re: Time zone in TIME_FORMAT

blahblah — Thu, 22 Oct 2015 23:06:55 GMT

Rather than submitting yet another question about timestamp formating, could you please update your answer with the correct format string to successfully parse a timestamp like 2015-10-22T22:41:52.546249+00:00? I tried %Y-%m-%dT%H:%M:%S.%6N%:z, but it doesn't seems to be working. The timezone info is probably not being captured as all event times are being translated as if the event timezone (always UTC) was the same as the splunk server (-0500), which, translated to my splunk user timezone (-0200), gives me 2015-10-23T01:41:52.546249-02:00. Any clue?

Re: Time zone in TIME_FORMAT

gjanders — Thu, 01 Sep 2016 08:17:29 GMT

After reading about %z on these pages and http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Configuretimestamprecognition , I found the python 3 documentation https://docs.python.org/3/library/datetime.html has this explanation:
"
%z UTC offset in the form +HHMM or -HHMM (empty string if the object is naive). (empty), +0000, -0400, +1030 (6)
%Z Time zone name (empty string if the object is naive). (empty), UTC, EST, CST

"

The documentation has been prompt updated to reflect this information correctly!
Furthermore, the preview functionality mentioned is now part of the "add data"/data upload option in Splunk...

Re: Time zone in TIME_FORMAT

woodcock — Sun, 23 Feb 2020 23:09:03 GMT

I made a request to ship your tool here:
https://ideas.splunk.com/ideas/EID-I-59

Re: Time zone in TIME_FORMAT

to4kawa — Sun, 23 Feb 2020 23:58:22 GMT

| makeresults 
| eval time1="01/Mar/2011:17:25:49.666+0000" 
| appendpipe 
    [ eval time2="2015-10-22T22:41:52.546249+00:00"] 
| eval _time=coalesce(strptime(time2,"%FT%T.%6Q%:z"),strptime(time1,"%d/%B/%Y:%T.%3Q%z"))

on Splunk ver8.0.1

btool [options] {check|validate-strptime|validate-regex}
Is this not enough?