Timezone not working in syslog

[erwinpastor]

I have setup a syslog forwarding (TCP and UDP) on few servers. The servers are using various time zones - New York, New Jersey, London and HK.

I've been trying to set all the timezone to local - Australia/Sydney but I can't get it work. Can anyone please help? I have the following setup in place:

in inputs.conf

[tcp://IP:2517]
disabled = false
sourcetype = syslog
index = index_name
host = host_name
connection_host = none
source = server_forwarder:tcp:2517
persistentQueueSize=500MB

[udp://IP:2517]
disabled = false
sourcetype = syslog
index = index_name
host = host_name
connection_host = none
no_appending_timestamp = true
source = server_forwarder:udp:2517
persistentQueueSize=500MB

In props.conf

[host::host_name]
TZ = Australia/Sydney
TZ_ALIAS = EST=GMT+10:00
TRANSFORMS-syslogseta = send_to_seta

Also tried without the alias

[host::host_name]
TZ = Australia/Sydney
TRANSFORMS-syslogseta = send_to_set

In transforms.conf

# sends logs out via syslog to SETA
[send_to_seta]
SOURCE_KEY = _MetaData:Index
REGEX = index_name
DEST_KEY = _SYSLOG_ROUTING
FORMAT = prod_seta_group

[Lucas_K]

A quick question, do you have a good reason for setting the timezone of those remote servers to local AU time rather than just leaving them alone?

In general, it is better to leave their timezones as the timezone from where the host from which the events are generated actually is.

The search head itself will translate those times to a locally displayed timezone as per their login preferences automatically. The raw event will still have their local time preserved.

[erwinpastor]

we have to translate all of the other region’s server syslog to Sydney time so that we can co-relate them for the events.

[Lucas_K]

You don't need to do that. The web interface will automatically do that if your time stamps and time zones are right.

ie. a search for events in the past hour will find both events from a sydney box @ 4:45pm 8/12/2014 AND london (5:45am 8/12/2014 local london time).

As your web interface is set to EDST you'll see both events with 4:45pm _time fields automatically (raw event time on the london event will be 5:45am).

[erwinpastor]

I did remove the TZ settings and I don't see the time fields in AU/Sydney time.

Sample search result for London server

12/8/14
6:14:28.800 AM  
2014-12-08 06:14:28,8 ... Common.Request-Timestamp=2014-12-08 06:13:26+00,
host = ... source = ... sourcetype = syslog

12/8/14 
6:14:28.800 AM  
2014-12-08 06:14:28,8  ... Common.Request-Timestamp=2014-12-08 06:12:56+00,
host = ... source = ... sourcetype = syslog

Search head server date:
[splunkp@ /]$ date
Mon Dec 8 17:16:47 EST 2014

Am I missing something?

[Lucas_K]

ahh want a field name as your time. This is totally different.

You have a couple of choices (unless someone has a better idea).

1. You can index them with that time as the _time field at index time via props.conf
so something like :

example.
[host::my_london_host]
TIME_PREFIX = Common.Request-Timestamp=
TIME_FORMAT = %Y-%m-%d %H:%M:%S%z

You could probably use a nice regex instead of the "Common.Request".

Pros: search time frame will catch all your events.
Cons: You've modified your original events.

OR

2. You can modify it at search time by renaming your Common.Request-Timestamp field to _time.
Example.
| eval new_time=strptime(Common.Request-Timestamp, "%Y-%m-%d %H:%M:%S%z") | eval new_epoch=strftime(new_time, "%s") | rename _time AS old_time | rename new_epoch AS _time
Pros: Original event is left intact.
Cons: Your base search will have to be large enough to include these events. You'll have to eval that time into epoch first and then rename to _time.

[erwinpastor]

Thanks Lucas K. I have overridden the date entry with something like you recommended and got the expected results.

[erwinpastor]

By the way, I also tried by source instead of host but it still don't work.

[source::server_forwarder:udp:2517]
TZ = Australia/Sydney

[source::server_forwarder:tcp:2517]
TZ = Australia/Sydney

[kml_uvce]

can you use btool and check if your configuration is working...

[erwinpastor]

Yeah, no unusual results when I run ./splunk btool check --debug