I have setup a syslog forwarding (TCP and UDP) on few servers. The servers are using various time zones - New York, New Jersey, London and HK.
I've been trying to set all the timezone to local - Australia/Sydney but I can't get it work. Can anyone please help? I have the following setup in place:
[tcp://IP:2517] disabled = false sourcetype = syslog index = index_name host = host_name connection_host = none source = server_forwarder:tcp:2517 persistentQueueSize=500MB [udp://IP:2517] disabled = false sourcetype = syslog index = index_name host = host_name connection_host = none no_appending_timestamp = true source = server_forwarder:udp:2517 persistentQueueSize=500MB
[host::host_name] TZ = Australia/Sydney TZ_ALIAS = EST=GMT+10:00 TRANSFORMS-syslogseta = send_to_seta Also tried without the alias [host::host_name] TZ = Australia/Sydney TRANSFORMS-syslogseta = send_to_set
# sends logs out via syslog to SETA [send_to_seta] SOURCE_KEY = _MetaData:Index REGEX = index_name DEST_KEY = _SYSLOG_ROUTING FORMAT = prod_seta_group
By the way, I also tried by source instead of host but it still don't work.
TZ = Australia/Sydney
TZ = Australia/Sydney
A quick question, do you have a good reason for setting the timezone of those remote servers to local AU time rather than just leaving them alone?
In general, it is better to leave their timezones as the timezone from where the host from which the events are generated actually is.
The search head itself will translate those times to a locally displayed timezone as per their login preferences automatically. The raw event will still have their local time preserved.
You don't need to do that. The web interface will automatically do that if your time stamps and time zones are right.
ie. a search for events in the past hour will find both events from a sydney box @ 4:45pm 8/12/2014 AND london (5:45am 8/12/2014 local london time).
As your web interface is set to EDST you'll see both events with 4:45pm _time fields automatically (raw event time on the london event will be 5:45am).
I did remove the TZ settings and I don't see the time fields in AU/Sydney time.
Sample search result for London server
12/8/14 6:14:28.800 AM 2014-12-08 06:14:28,8 ... Common.Request-Timestamp=2014-12-08 06:13:26+00, host = ... source = ... sourcetype = syslog 12/8/14 6:14:28.800 AM 2014-12-08 06:14:28,8 ... Common.Request-Timestamp=2014-12-08 06:12:56+00, host = ... source = ... sourcetype = syslog
Search head server date:
[splunkp@ /]$ date
Mon Dec 8 17:16:47 EST 2014
Am I missing something?
ahh want a field name as your time. This is totally different.
You have a couple of choices (unless someone has a better idea).
1. You can index them with that time as the _time field at index time via props.conf
so something like :
TIMEPREFIX = Common.Request-Timestamp=
TIMEFORMAT = %Y-%m-%d %H:%M:%S%z
You could probably use a nice regex instead of the "Common.Request".
Pros: search time frame will catch all your events.
Cons: You've modified your original events.
2. You can modify it at search time by renaming your Common.Request-Timestamp field to time.
| eval newtime=strptime(Common.Request-Timestamp, "%Y-%m-%d %H:%M:%S%z") | eval newepoch=strftime(newtime, "%s") | rename time AS oldtime | rename new_epoch AS _time
Pros: Original event is left intact.
Cons: Your base search will have to be large enough to include these events. You'll have to eval that time into epoch first and then rename to _time.