topic Timezone change TZ not working. in Getting Data In

Timezone change TZ not working.

knalla — Wed, 30 Sep 2020 01:27:37 GMT

Hi,

We are getting logs in UTC format, I tried using TZ=UTC on Heavy forwarder, but its not working for all events.

Working events:
_time Event_Timestamp
2019-07-26 08:25:44.000 2019-07-26T13:24:43.3570000Z
2019-07-26 07:21:45.000 2019-07-26T12:20:42.0070000Z

Not working:
_time Event_Timestamp
2019-07-26 07:04:06.417 2019-07-26T07:04:06.4170000Z
2019-07-26 07:03:06.293 2019-07-26T07:03:06.2930000Z

Any suggestions?

Re: Timezone change TZ not working.

richgalloway — Wed, 30 Sep 2020 01:27:39 GMT

What are your TIME_PREFIX and TIME_FORMAT settings?

Re: Timezone change TZ not working.

knalla — Fri, 26 Jul 2019 17:06:35 GMT

nothing, just added TZ=UTC for source type.

Re: Timezone change TZ not working.

knalla — Wed, 30 Sep 2020 01:27:42 GMT

Do I need TIME_PREFIX and TIME_FORMAT?

Re: Timezone change TZ not working.

knalla — Wed, 30 Sep 2020 01:27:44 GMT

Added these in props.conf

TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
TIME_PREFIX = \"TimeStamp\": \"
TZ = UTC

Re: Timezone change TZ not working.

richgalloway — Wed, 30 Sep 2020 01:27:58 GMT

You don't need TIME_PREFIX and TIME_FORMAT, but it's a Best Practice to use both.
There's no need for the escape characters in TIME_PREFIX.
TIME_FORMAT should include the time zone indicator. %Y-%m-%dT%H:%M:%S.%7N%Z

Re: Timezone change TZ not working.

woodcock — Sat, 27 Jul 2019 14:50:28 GMT

Do not set TZ with TZ=, set it with:

TIME_PREFIX = ^\d+\-\d+\-\d+\s+\d+:\d+:\d+\.\d+\s+
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%7N%Z