Where Is Timezone Offset Information on Universal Forwarder?

[stevepraz]

Trying to determine why some of my forwarders sending in data from Windows virtual desktop instances are having their data offset at the indexer and others are not. I know the documentation says that post 6.0 infrastructure will respect the timezone information dictated by the forwarder.

Where is this information specified on the forwarder? I don't remember configuring anything like that. Is it something that the installer obtains from the local machine at install time? I'm trying to confirm if this setting is in place on the forwarder or if the issue with the offset is occurring on the indexer side.

[woodcock]

There was a change added to v6.?.? so that, for the first time ever, a TZ= setting in an indexer's props.conf file has the HIGHEST precedence. It is possible that some have this value and some don't.

[Jeremiah]

The forwarder timezone is obtained automatically from the system (its not something you need to configure). When you run a search, look for the the date_zone field. It will contain the offset applied to your forwarders. When troubleshooting timezone issues, I usually check the following:

1. Is the zone info included in the event and is it getting processed properly?
2. Is the forwarder running 6.0+? Make sure negotiateNewProtocol is not set to false in outputs.conf (uncommon)
3. Is there a props.conf stanza on the indexer explicitly setting the timezone for either the host, source, or sourcetype?

If you have a mix of universal forwarders that seem to be working and some not, I'd go through this list and see if any of the differences above apply.

[stevepraz]

Thanks for the clarification. I don't think the timezone data is in the event (as in, it isn't specified in the text of the log file line). Forwarders are running 6.0+ and do not have the outputs.conf config you mentioned. Also, I don't believe any of the props.conf on the indexers have the configs you mention.

When does the timezone get obtained on the forwarder? At startup? If the timezone on the machine changes while the forwarder is running, does it know that?

Here's some info from my virtual desktop administrator (we are running these forwarders on virtual desktops) about how these machines behave:

There were approx. 11 desktops that
were logging the wrong time. I do
wonder if the splunk forwarder is
getting confused when the view agent
changes the timezone with the
logon/logoff activity. The view agent
on the virtual desktop will change the
timezone to match what's on the
physical desktop. The timezone is
changed back to be in sync with the
ESXi host when no one is logged on.