Getting Data In

Bad default system timezone recognition

Communicator

I am indexig aix_audit data from my splunk instance (AIX)

The servers timezone seems to be ok - Tue Oct 9 17:08:02 GMT+02:00 2012 and the user is configured to use default system timezone (should be GMT+02:00).

But when I search the data it shows a wrong timestamp althought datezone field is local.

10/9/12 3:11:34.000 Automatic Timestamp
Tue Oct 09 17:11:34 2012 Date in log.

A simple workaround is just to change the user's timezone to the correct one but I would like to know where the problem lies.

Thanks.

Tags (3)
1 Solution

Splunk Employee
Splunk Employee

When you run into problems with timestamp recognition out of the box, the proper course of action is to provide Splunk with a set of instructions telling it where the time stamp in the event is, the format it follows, and how long the time stamp is within the event. These settings are TIME_PREFIX, TIME_FORMAT, and MAX_TIMESTAMP_LOOKAHEAD. These steps are outlined here with examples:

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Configuretimestamprecognition

View solution in original post

Communicator

Does anyone know how to check the default system timezone used by Splunk in the server? Something more precise than local.

I am still thinking is due to bad automatic user's timezone association.

Communicator

Did you ever figure this out? The 'Accepted Answer' does not seem to address the problem. I am seeing this in 6.4.1. A few users are being shown the wrong timezone, although their accounts are set to use Default System Timezone. The workaround is to adjust the timezone to the suit what Splunk is showing them, but it does not explain why Splunk is misinterpreting the timezone

0 Karma

Communicator

Found a search head with time drift of almost an hour. pulled the sh out of the cluster, fixed the time, ran splunk clean on it and put it back in rotation. no more timezone issue reported by users

0 Karma

Contributor

Splunk does advise to have some sort of time sync (usually using an NTP server) solution for all servers in the farm to make sure time stamps are always consistent.

0 Karma

Communicator

Yes, absolutely our bad on that. The way it manifests itself in the UI is really strange, though. It's difficult to research due to so many questions and documentation related to timezone configuration on the server side.

0 Karma

Splunk Employee
Splunk Employee

When you run into problems with timestamp recognition out of the box, the proper course of action is to provide Splunk with a set of instructions telling it where the time stamp in the event is, the format it follows, and how long the time stamp is within the event. These settings are TIME_PREFIX, TIME_FORMAT, and MAX_TIMESTAMP_LOOKAHEAD. These steps are outlined here with examples:

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Configuretimestamprecognition

View solution in original post

Communicator

In this case if Splunk uses the time zone of the Splunk server that indexes the event, why all information has a offset of -2 hours included internal indexes?

Is seems everything is configurated in GMT not GMT+2

0 Karma

Motivator

Splunk does not index all data in GMT. It indexes all data in whatever timezone the indexer is set to.

Search results are displayed in the current users local time. This can be changed by clicking on their name in the upper right to get to their preferences. This is display only, and does not modify the underlying data.

EDIT: documentation on how timezone is applied to events at index time
http://docs.splunk.com/Documentation/Splunk/5.0/data/Applytimezoneoffsetstotimestamps

Communicator

I agree but it happens with all the data indexed, for example, predefinied inputs from splunk app for Unix and Linux.

Splunk index all data in GMT and the problem seems to be that splunk dont apply the default system timezone to show the correct time to the user.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!