topic Re: How can I get now() to be in UTC? in Dashboards & Visualizations

How can I get now() to be in UTC?

cwapshere — Mon, 03 Oct 2016 22:17:33 GMT

Seems like such a trivial question but I can't find the answer.

I'm new to Splunk and troubleshooting a dashboard set up by someone else. He was using "now() - _time" to show "Data Freshness" - however _time is in UTC in the incoming data, so I just need now() to be in UTC as well. How do I do that in the search?

Re: How can I get now() to be in UTC?

sundareshr — Mon, 03 Oct 2016 23:29:07 GMT

Try something like this.

| makeresults | eval x=now() | eval pst=strptime(strftime(x,"%m/%d/%Y %H:%M:%S UTC"),"%m/%d/%Y %H:%M:%S %Z") | eval mst=strptime(strftime(x,"%m/%d/%Y %H:%M:%S MST"),"%m/%d/%Y %H:%M:%S %Z")  | eval est=strptime(strftime(x,"%m/%d/%Y %H:%M:%S EST"),"%m/%d/%Y %H:%M:%S %Z") | eval utc=strptime(strftime(x,"%m/%d/%Y %H:%M:%S UTC"),"%m/%d/%Y %H:%M:%S %Z") | table x pst mst est utc | foreach * [eval <<FIELD>>=strftime(<<FIELD>>, "%c")]

Re: How can I get now() to be in UTC?

cwapshere — Tue, 04 Oct 2016 00:07:06 GMT

Thanks for that. I tried a simplified version which just ave me the UTC time I wanted:

eval nowutc=strptime(strftime(now(),"%m/%d/%Y %H:%M:%S UTC"),"%m/%d/%Y %H:%M:%S %Z") | eval result = nowutc - _time

But the resultant number is the same as I get from this:

eval result=now() - _time

I'm starting to think the issue is _time and not now(). Eg., I have a panel showing "Last Event Was 27 Hours Ago" when I have events from 16 hours ago - and I am in UTC+11. All the data files are in JSON format with a _time field, for every event, in UTC. Now I'm thinking splunk is interpreting that as local time...

Re: How can I get now() to be in UTC?

sundareshr — Tue, 04 Oct 2016 00:15:27 GMT

Have you tried the reltime command? It may give you what you're looking for

https://docs.splunk.com/Documentation/Splunk/6.4.3/SearchReference/Reltime

_time is always stored as epoch time and then converted to the timezone specified in .conf files.

Re: How can I get now() to be in UTC?

sideview — Tue, 04 Oct 2016 00:17:29 GMT

It's kind of a mindbending question, but I think the answer is this - _time is an epochtime value, and that's also what now() returns. ie a number of seconds since the "epoch" started, which is in turn defined as 12am on 1/1/1970 if you were standing in GMT at that moment.

The Splunk UI itself will automatically convert _time to be in local time, and this can lead to a misunderstanding that _time is itself a string value. Which can in turn lead to confusing attempts to convert it to another timezone like UTC.

hth

Re: How can I get now() to be in UTC?

cwapshere — Tue, 04 Oct 2016 01:11:01 GMT

I think there may be an issue with the splunk forwarder assuming the incoming _time values are local instead of UTC.

I'm going to see if I can figure out how to get at the raw value because I can't change the forwarder at this time. Thanks for the help!

Re: How can I get now() to be in UTC?

lguinn2 — Tue, 04 Oct 2016 01:44:46 GMT

I think you have most of this from the other answers, but let me summarize:

When the data comes from a forwarder, the forwarder (version 6.x) supplies local time zone information that Splunk uses to calculate _time in UTC. So, the data is always stored in the index with _time in UTC. If the data does not have the proper time, it may be because the Splunk admin who set up the forwarding missed something. Perhaps the time zone information is not being picked up somewhere along the way, or props.conf needs to have a timezone setting on the indexer. You can find more info here: How timestamp assignment works

That said - when you sign onto Splunk, there is a time zone associated with your user account. So if you had your user timezone set to Pacific Time, then the Splunk UI will display all times (including _time and now()) in Pacific Time. So what you see may well be different from the value that is actually stored, but it will be consistent.

| eval freshness = now() - _time

Calculates "freshness" as the number of seconds between the event timestamp and the time that the search started.

Finally, you might also try the reltime command for what you want. (If I understand what you want.)

yoursearchhere
| head 1
| reltime 
| eval output="Last event was " . reltime
| table output

Creates a field named "output" which displays stuff like "Last event was 27 seconds ago." And you don't have to do the math yourself.

Re: How can I get now() to be in UTC?

cwapshere — Tue, 04 Oct 2016 02:28:55 GMT

Wow thanks for the great explanation! I know sundareshr had also suggested reltime but when I looked at the doc I couldn't understand why it had no argumates - was still thinking I needed to feed it two dates. That does seem to have worked.

Re: How can I get now() to be in UTC?

sideview — Tue, 04 Oct 2016 02:40:45 GMT

One minor note - @lguinn said "the data is always stored in the index with _time in UTC", but this is confusing and not technically true. Splunk stores the _time in "epochtime" , meaning as the number of seconds since 12am 1/1/1970 in gmt.
On the other hand "UTC" is a timezone, so to store time "in UTC" would technically mean to store it as a string that looks something like "Tue, 04 Oct 2016 02:38:31 UTC".