Getting Data In

Display time in UTC

rtkelly
Explorer

We are trying to run a report that groups data by the UTC date of events occur. Our Heavy forwarders collect the data from different regions and correctly set the TZ field according to the time fields from the source data. We can tell that this is correct, because the value of the _time field is the epoch time of the events in UTC.

When we run the report, we use strftime(_time, "%Y-%m-%d") to get the date that the event occurred. The problem is that this appears to be reporting the date according to the local representation of the indexer. We know this, because if we add %z to the time format it shows different timezones for each indexer. If we add a map function like "stats" to the command prior to computing the strftime we get the timezone of the search head.

What we want to do is to create a string that represent the UTC date of the event regardless of what timezone the indexer or search head is running on. Since this is a very common scenario for us, is there a configuration setting or search function that can ensure that UTC time is used to render the date?

Tags (2)
0 Karma
1 Solution

lguinn2
Legend

One way is to set the timezone on your indexers and search heads to UTC. Do this in the OS, and Splunk will render the timezone in UTC by default.

In Splunk 4.3, each user can choose their own timezone for viewing the data/reports/etc. Go to Manager » Access controls » Users to set this for users, or to Manager » Your account to set the timezone for yourself.

Note that Splunk always stores the data in UTC in the index, but displays it according to the indexer's TZ or the users's TZ.

View solution in original post

0 Karma

bhatti009
New Member

Little late but you can try

...
| eval offset=strftime(_time,"%z")
| eval _time=_time-(offset*36)
0 Karma

asieira
Path Finder

I was faced with the same problem recently and I solved it by writing the following macro:

[strftime_utc(2)]
args = field, format
definition = "strftime($field$ - (strptime(strftime($field$, \"%Y-%m-%dT%H:%M:%SZ\"), \"%Y-%m-%dT%H:%M:%S%Z\")-strptime(strftime($field$, \"%Y-%m-%dT%H:%M:%S\"), \"%Y-%m-%dT%H:%M:%S\")), \"$format$\")"
iseval = 1

So you can now write a search that looks like this:

index=main | eval utc_time=`strftime_utc(_time, "%Y-%m-%dT%H:%M:%SZ")`

Regardless of what the timezone is on each event, this will cause the output to be in UTC.

Thanks to @richgalloway for the initial suggestion that lead to this.

lguinn2
Legend

One way is to set the timezone on your indexers and search heads to UTC. Do this in the OS, and Splunk will render the timezone in UTC by default.

In Splunk 4.3, each user can choose their own timezone for viewing the data/reports/etc. Go to Manager » Access controls » Users to set this for users, or to Manager » Your account to set the timezone for yourself.

Note that Splunk always stores the data in UTC in the index, but displays it according to the indexer's TZ or the users's TZ.

View solution in original post

0 Karma

landen99
Motivator

I think that splunk needs a search line command to override the time zone settings in the client for that search, much like "earliest" and "latest" overrides the UI time settings for the search.

0 Karma

lguinn2
Legend

No you can't set timezone by app - sorry. I think it's great if you can have all your servers in UTC - it simplifies management from the OS across the whole software stack. But that's just a personal preference.

0 Karma

rtkelly
Explorer

Thanks. We were thinking about switching all the servers to run in UTC time. We were also thinnking of upgraing to 4.3 soon. We'd like to run the scheduled searches from app context. Would there be a preferred timezone for an app?

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!