Getting Data In

Display time in UTC

Explorer

We are trying to run a report that groups data by the UTC date of events occur. Our Heavy forwarders collect the data from different regions and correctly set the TZ field according to the time fields from the source data. We can tell that this is correct, because the value of the _time field is the epoch time of the events in UTC.

When we run the report, we use strftime(_time, "%Y-%m-%d") to get the date that the event occurred. The problem is that this appears to be reporting the date according to the local representation of the indexer. We know this, because if we add %z to the time format it shows different timezones for each indexer. If we add a map function like "stats" to the command prior to computing the strftime we get the timezone of the search head.

What we want to do is to create a string that represent the UTC date of the event regardless of what timezone the indexer or search head is running on. Since this is a very common scenario for us, is there a configuration setting or search function that can ensure that UTC time is used to render the date?

Tags (2)
0 Karma
1 Solution

Legend

One way is to set the timezone on your indexers and search heads to UTC. Do this in the OS, and Splunk will render the timezone in UTC by default.

In Splunk 4.3, each user can choose their own timezone for viewing the data/reports/etc. Go to Manager » Access controls » Users to set this for users, or to Manager » Your account to set the timezone for yourself.

Note that Splunk always stores the data in UTC in the index, but displays it according to the indexer's TZ or the users's TZ.

View solution in original post

0 Karma

Path Finder

I was faced with the same problem recently and I solved it by writing the following macro:

[strftime_utc(2)]
args = field, format
definition = "strftime($field$ - (strptime(strftime($field$, \"%Y-%m-%dT%H:%M:%SZ\"), \"%Y-%m-%dT%H:%M:%S%Z\")-strptime(strftime($field$, \"%Y-%m-%dT%H:%M:%S\"), \"%Y-%m-%dT%H:%M:%S\")), \"$format$\")"
iseval = 1

So you can now write a search that looks like this:

index=main | eval utc_time=`strftime_utc(_time, "%Y-%m-%dT%H:%M:%SZ")`

Regardless of what the timezone is on each event, this will cause the output to be in UTC.

Thanks to @richgalloway for the initial suggestion that lead to this.

Legend

One way is to set the timezone on your indexers and search heads to UTC. Do this in the OS, and Splunk will render the timezone in UTC by default.

In Splunk 4.3, each user can choose their own timezone for viewing the data/reports/etc. Go to Manager » Access controls » Users to set this for users, or to Manager » Your account to set the timezone for yourself.

Note that Splunk always stores the data in UTC in the index, but displays it according to the indexer's TZ or the users's TZ.

View solution in original post

0 Karma

Motivator

I think that splunk needs a search line command to override the time zone settings in the client for that search, much like "earliest" and "latest" overrides the UI time settings for the search.

0 Karma

Legend

No you can't set timezone by app - sorry. I think it's great if you can have all your servers in UTC - it simplifies management from the OS across the whole software stack. But that's just a personal preference.

0 Karma

Explorer

Thanks. We were thinking about switching all the servers to run in UTC time. We were also thinnking of upgraing to 4.3 soon. We'd like to run the scheduled searches from app context. Would there be a preferred timezone for an app?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!