topic Re: The timestamp has a one hour offset in Getting Data In

The timestamp has a one hour offset

djoiret — Thu, 28 Oct 2021 14:27:09 GMT

Hello,

I am using "Splunk_TA_juniper" and I noticed a new problem with timestamp: there is a one hour offset for the timestamp compared to the time in the event. For instance, when I have an event whose _raw value starts with "Oct 28 15:12:37 fw-01-gra RT_FLOW: ...", the timestamps is "2021-10-28T16:12:37.000+02:00" (16h instead of 15h). In addition, the event will only appear after an hour after its received by the indexer, in fact when the timestamp value is less than the current time.

This behaviour is new. When I examine events for september (for instance), the timestamp matches the time in the event.

I tried to restart Splunk and the forwarder, nothing was changed. I haven't modify the configuration files for a long time, and I don't know what to do.

Do you have an idea of what is going on or a possible solution?

Regards

Denis

Re: The timestamp has a one hour offset

richgalloway — Thu, 28 Oct 2021 16:46:03 GMT

It sounds like the country where the source resides recently changed from Summer time to standard time, but the source is still reporting timestamps in Summer time. It could be the application at fault or a Splunk setting. Can you share the props.conf settings for that sourcetype?

Re: The timestamp has a one hour offset

djoiret — Tue, 02 Nov 2021 16:47:43 GMT

Hello,

After some other investigations, I noticed that in fact ALL events having syslog format for date and time had this 1 hour offset. The problem started on 10/28/2021 00:00:00 until 10/30/2021 23:59:59. All events after 10/31/2021 00:00:00 have a correct timestamp, so the problem just disappeared.

In France we changed from summer time to winter time on Sun 10/31/2021 at 03:00:00 (at 03:00:00, local time was changed to 02:00:00).

I never noticed this problem before, I have been using Splunk since 2014. In my opinion, this looks like a bug in Splunk. I know no algorithm for changing summer time to winter time taking place on tuesday.

For the application Splunk_TA_juniper, there was no parameters about time in "default/props.conf". There are no parameter either in system props.conf.

I added these parameters in "local/props.conf" of application Splunk_TA_juniper before my previous post :

[juniper:junos:firewall]
TIME_FORMAT = "%b %e %T"
TZ = Europe/Paris

This did not change anything.

Denis

Re: The timestamp has a one hour offset

isoutamo — Tue, 02 Nov 2021 20:52:31 GMT

Hi
I'm not sure if this related anything in your case, but we have noticed that there are still some network equipments, which cannot handle timestamps correctly with syslog feed when summertime starts or ends. Usually it has require reboot and time by time even this haven't helps. There could be two different timestamp in event and one is right and second one is not.
r. Ismo

Re: The timestamp has a one hour offset

djoiret — Wed, 03 Nov 2021 08:18:15 GMT

Hello,

The problem is not from the sources. I have several logs sources (Juniper logs, "authpriv" from Linux servers, Cisco ESA logs, etc.) and all had this 1 hour offset between 10/28 and 10/31 02:00:00. You can see in the joined image two events, one just before 02:00:00 (at 01:59:59) and the second one at 02:00:00. The timestamp for the first event is wrong, the timestamp for the second event is correct.

Denis