topic Re: Monitor logs from Different Time Zones in Splunk Search

How to monitor logs from Different Time Zones?

sarahnazzar — Wed, 04 May 2022 16:34:00 GMT

Hello Splunkers!

Initially I added the monitor stanza for all the inputs from various time zones and then when I had a check there was difference _time and the time present in the event and there was a lag by 1 or 2 hours based on that country's time zone and Splunk time zone, then figured out the it is because Splunk looks for a timestamp in the event and parse the data. Now , I need to monitor logs being received from different time zones from various countries and Splunk is in different time zone, can you please drop in your knowledge on this please.

When investigated, found that we can add the below as false as per https://docs.splunk.com/Documentation/Splunk/8.2.6/Admin/Propsconf

BREAK_ONLY_BEFORE_DATE = <boolean>
DATETIME_CONFIG = NONE

And could see that there are options to define the time zones using TZ. Can anyone help me out please!

Example:
My source:

test.csv
SYSTEMDATE,SYSTEMTIME,FAILUREMESSAGE

"2022-05-04","12.51.08", The JobA has failed

"2022-05-04","13.00.05", The JobB has failed

Data reflecting in Splunk UI:

Time	Event
04/05/2022 12:51:03.000	SYSTEMDATE,SYSTEMTIME,FAILUREMESSAGE
04/05/2022 11:51:08.000	"2022-05-04","14.51.08",The JobA has failed
04/05/2022 12:00:05.000	"2022-05-04","13.00.05",The JobB has failed

Only the below event is reflecting at the current time when the job is triggered from Application end which is the correct one since the below has no timestamp defined.

04/05/2022
12:51:03.000

SYSTEMDATE,SYSTEMTIME,FAILUREMESSAGE

Source time zone: Various Countries like Italy, Romania, Cyprus etc.,

Destination/Splunk Time Zone: BST

Many thanks!

Sarah

Re: Monitor logs from Different Time Zones

richgalloway — Wed, 04 May 2022 14:58:34 GMT

For each forwarder that is sending these logs, add a TZ setting to the appropriate props.conf stanza. The forwarder will tell the indexers the correct time zone to use.

Re: Monitor logs from Different Time Zones

sarahnazzar — Thu, 05 May 2022 05:07:27 GMT

@richgalloway Thanks for your response!

Correct Time zone in the sense it will be using the timezone configured in Splunk right i.e., BST current time when the data comes in.

For example if the time zone is of Romania then will TZ = EET work under that particular sourcetype's props.conf

[jobcsv]

TZ = EET

Had a check in TZ database but couldn't find the same, can you please help me out?

https://en.m.wikipedia.org/wiki/List_of_tz_database_time_zones

Re: Monitor logs from Different Time Zones

richgalloway — Thu, 05 May 2022 12:57:11 GMT

Yes, by "correct time zone" I mean the one configured.

You should be able to use TZ=EET or TZ=Europe/Bucharest in props.conf.

Re: Monitor logs from Different Time Zones

sarahnazzar — Fri, 06 May 2022 17:08:44 GMT

@richgalloway Many thanks that worked!! 😊