I have a scenario where I am indexing event logs from Windows servers across 5 different time zones:
All of the servers are sending their security log to forwarders and from there they are sent to the indexers. So I've got an issue where when I search logs in real time nothing is showing up, I've got to look forwarder 2 hours or backward 2 hours to see logs coming in. I'm trying to figure out how on a forwarder I can set the TZ for specific servers. Looking at http://docs.splunk.com/Documentation/Splunk/4.1.8/admin/ApplyTimezoneOffsetstotimestamps, it looks like it should be simple but I can't seem to figure it out.
Is there something I can put on the UniversalForwarder config or on the Indexer that will allow me to apply a Time Zone to specific servers? Reading the article above it seems to suggest the Time Zone should be that of the server sending the data. But they all hit the props stanza [WinEventLog://Security] so I'm a bit stuck.
My forwarder configuration is as follows.
[splunktcp://9997] disabled = 0
[WinEventLog:Security] TRANSFORMS-security = security_null, security_truncate, security_index
[security_null] REGEX = . DEST_KEY = queue FORMAT = nullQueue [security_truncate] REGEX = (?msi)(.*)(This event is generated|Certificate information is only provided) DEST_KEY = _raw FORMAT = $1 [security_index] REGEX = (?msi)^EventCode=(list of event Id's removed for brevity)\D DEST_KEY = queue FORMAT = indexQueue
Everything is working excellent, except how can I specify a different timezone for different servers in the props.conf if the stanza is looking for [WinEventLog:Security]?
The configuration for the servers is all identical.
[WinEventLog://Security] checkpointInterval = 5 current_only = 1 disabled = 0 index = security start_from = oldest
[tcpout] defaultGroup = forwarder_9997 [tcpout:forwarder_9997] server = forwarder:9997 [tcpout-server://forwarder:9997]
Well it seems to be working at the moment.
[host::PERTH*] TZ = Australia/Perth
So the server is this time zone is now reporting the right times. But...another server in another TZ which is still on the default stanza in props.conf is not having any issues either. So I can't tell if it was just a case of the forwarder and/or indexer playing catch up or if the setting fixed it.