Getting Data In
Highlighted

Timezone not working in syslog

Explorer

I have setup a syslog forwarding (TCP and UDP) on few servers. The servers are using various time zones - New York, New Jersey, London and HK.

I've been trying to set all the timezone to local - Australia/Sydney but I can't get it work. Can anyone please help? I have the following setup in place:

in inputs.conf

[tcp://IP:2517]
disabled = false
sourcetype = syslog
index = index_name
host = host_name
connection_host = none
source = server_forwarder:tcp:2517
persistentQueueSize=500MB

[udp://IP:2517]
disabled = false
sourcetype = syslog
index = index_name
host = host_name
connection_host = none
no_appending_timestamp = true
source = server_forwarder:udp:2517
persistentQueueSize=500MB

In props.conf

[host::host_name]
TZ = Australia/Sydney
TZ_ALIAS = EST=GMT+10:00
TRANSFORMS-syslogseta = send_to_seta

Also tried without the alias

[host::host_name]
TZ = Australia/Sydney
TRANSFORMS-syslogseta = send_to_set

In transforms.conf

# sends logs out via syslog to SETA
[send_to_seta]
SOURCE_KEY = _MetaData:Index
REGEX = index_name
DEST_KEY = _SYSLOG_ROUTING
FORMAT = prod_seta_group
0 Karma
Highlighted

Re: Timezone not working in syslog

Explorer

By the way, I also tried by source instead of host but it still don't work.

[source::server_forwarder:udp:2517]
TZ = Australia/Sydney

[source::server_forwarder:tcp:2517]
TZ = Australia/Sydney

0 Karma
Highlighted

Re: Timezone not working in syslog

Builder

can you use btool and check if your configuration is working...

0 Karma
Highlighted

Re: Timezone not working in syslog

Explorer

Yeah, no unusual results when I run ./splunk btool check --debug

0 Karma
Highlighted

Re: Timezone not working in syslog

Motivator

A quick question, do you have a good reason for setting the timezone of those remote servers to local AU time rather than just leaving them alone?

In general, it is better to leave their timezones as the timezone from where the host from which the events are generated actually is.

The search head itself will translate those times to a locally displayed timezone as per their login preferences automatically. The raw event will still have their local time preserved.

0 Karma
Highlighted

Re: Timezone not working in syslog

Explorer

we have to translate all of the other region’s server syslog to Sydney time so that we can co-relate them for the events.

0 Karma
Highlighted

Re: Timezone not working in syslog

Motivator

You don't need to do that. The web interface will automatically do that if your time stamps and time zones are right.

ie. a search for events in the past hour will find both events from a sydney box @ 4:45pm 8/12/2014 AND london (5:45am 8/12/2014 local london time).

As your web interface is set to EDST you'll see both events with 4:45pm _time fields automatically (raw event time on the london event will be 5:45am).

0 Karma
Highlighted

Re: Timezone not working in syslog

Explorer

I did remove the TZ settings and I don't see the time fields in AU/Sydney time.

Sample search result for London server

12/8/14
6:14:28.800 AM  
2014-12-08 06:14:28,8 ... Common.Request-Timestamp=2014-12-08 06:13:26+00,
host = ... source = ... sourcetype = syslog

12/8/14 
6:14:28.800 AM  
2014-12-08 06:14:28,8  ... Common.Request-Timestamp=2014-12-08 06:12:56+00,
host = ... source = ... sourcetype = syslog

Search head server date:
[splunkp@ /]$ date
Mon Dec 8 17:16:47 EST 2014

Am I missing something?

0 Karma
Highlighted

Re: Timezone not working in syslog

Motivator

ahh want a field name as your time. This is totally different.

You have a couple of choices (unless someone has a better idea).

1. You can index them with that time as the _time field at index time via props.conf
so something like :

example.
[host::mylondonhost]
TIMEPREFIX = Common.Request-Timestamp=
TIME
FORMAT = %Y-%m-%d %H:%M:%S%z

You could probably use a nice regex instead of the "Common.Request".

Pros: search time frame will catch all your events.
Cons: You've modified your original events.

OR

2. You can modify it at search time by renaming your Common.Request-Timestamp field to time.
Example.
| eval new
time=strptime(Common.Request-Timestamp, "%Y-%m-%d %H:%M:%S%z") | eval newepoch=strftime(newtime, "%s") | rename time AS oldtime | rename new_epoch AS _time
Pros: Original event is left intact.
Cons: Your base search will have to be large enough to include these events. You'll have to eval that time into epoch first and then rename to _time.

Highlighted

Re: Timezone not working in syslog

Explorer

Thanks Lucas K. I have overridden the date entry with something like you recommended and got the expected results.

0 Karma