Getting Data In

Standard timezone recognition in Splunk

leatherface
Explorer

We are pulling in log lines from our customers from a wide variety of time zones. The lines start with:

INFO  : 16 Dec 2013 09:49:47,123 AKST

Note that we are getting the logs in via the STOMP protocol and plugin, so there is no Universal Forwarder to configure - the STOMP plugin just pulls the logs as individual lines of text.

So the issue we are experiencing is that with some of the more unusual (but still totally standard) time zones, Splunk is failing to understand the time zone so that data ends up being imported in the future or the past (in the example above, it thinks the Alaskan Standard Time zone is GMT-1 rather than GMT-8). From various searches I understand I can set a TZ_ALIAS for these edge cases. However, it seems a little odd that Splunk only understands a subset of all standard time zone acronyms - am I missing something to get this working out-of-box for all time zones. At the very least. is there a way I can find out which time zones Splunk will/will not understand (so I can set them up ahead of time)?

Thanks in advance for any and all help!

Tags (1)
0 Karma

dbylertbg
Path Finder

The official answer can be found here, quoted below:

zoneinfo (TZ) database

The zoneinfo database is a publicly
maintained database of time zone
values.

UNIX versions of Splunk rely on a TZ database included with the UNIX
distribution you're running on. Most
UNIX distributions store the database
in the directory: /usr/share/zoneinfo.

Solaris versions of Splunk store TZ information in this directory:
/usr/share/lib/zoneinfo.

Windows versions of Splunk ship with a copy of the TZ database.

Refer to the zoneinfo (TZ) database
for all permissible TZ values.

Unfortunately the Wikipedia page the docs link to is less than helpful as it doesn't actually list the acceptable abbreviations for each time zone. This article lists abbreviations and includes AKST... but it's still not clear if this list is one that Splunk considers valid.

Since I'm running on linux, I decided to check /usr/share/zoneinfo/America/Anchorage. But that's not helpful either... it's a binary file.

Searching the contents of $SPLUNK_HOME for some common timezone abbreviations led me to the file:

$Splunk_Home/etc/datetime.xml

... whose header comment states:

<!-- This file contains the general formulas for parsing date/time formats. -->

Starting at line 49 it contains the following list:

<define name="_zone" extract="zone">
     <text><![CDATA[((?:(?:UT|UTC|GMT(?![+-])|CET|CEST|CETDST|MET|MEST|METDST|MEZ|MESZ|EET|EEST|EETDST|WET|WEST|WETDST|MSK|MSD|IST|JST|KST|HKT|AST|ADT|EST|EDT|CST|CDT|MST|MDT|PST|PDT|CAST|CADT|EAST|EADT|WAST|WADT|Z)|(?:GMT)?[+-]\d\d?:?(?:\d\d)?)(?!\w))?]]></text>
</define>

So... I can't say for sure but it looks like this may be the list of timezones Splunk will automatically recognize??? I'll also submit feedback to the doc team asking for clarification and linking to this thread.

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...