Using props.conf on SplunkUniversalForwarder to denote TimeZone

[dan_ce]

TimeZone specification in props.conf on a SplunkUniversalForwarder instance does not appear to be working for me.

• SplunkUniversalForwarder instance version 6.3.2
• Splunk instance (indexer) version 7.0.0
• The application server running the forwarder is in US/Eastern system timezone (cannot change)
• The logs are generated in UTC without a timezone specifier in the string (cannot change)

As the logs are received by Splunk they are interpreted as being UTC-5 as I suppose the forwarder is appending its system timezone. As the _time field is subsequently converted to UTC we see logs with time values 5 hours in the future.

I want to configure the forwarder instance to explicitly state that the timezone of the records it's sending on is UTC. I've tried the following:

props.conf in:
- apps/appname/local
- apps/appname/default
- system/local
- system/default

I've tried several different stanzas to match the log monitors, for example:

[sourcetype]
TZ = UTC

[host::hostname*]
TZ = UTC

[source::...//logs//debug_*]
TZ = UTC

[default]
TZ = UTC

All to no avail. Actually I am now at the point where I don't think the configuration is a problem, but it may still be. I don't see any difference to the logs imported regardless of which of the above options I use, so it's like it's being overridden at the indexer or simply not picked up.

Documentation suggests that the forwarder should be able to append TimeZone information from props.conf post version 6 and that this ought to be respected when indexed. I'm not seeing this behaviour at all. I don't want to / can't configure this at the indexer as I have servers in multiple different timezones, they each need to be able to specify the source tz information.

Can anyone suggest any other avenues of exploration? Thanks in advance.

[richgalloway · Accepted solution]

The props.conf file should go on the indexers rather than the universal forwarders. If the sourcetype has forwarders in different time zones, then use a heavy forwarder and put the props.conf file there.

[dan_ce]

Thank you for your response. I must have misunderstood the line in the documentation which discusses timezone application precedence which states:

"If the forwarder and the receiving indexer are version 6.0 or later, use the time zone that the forwarder provides."

Is it then the case that this is hardwired to use only the system timezone of the instance on which the Universal Forwarder sits? It's impossible to modify this using props.conf?

[ddrillic]

-- Is it then the case that this is hardwired to use only the system timezone of the instance on which the Universal Forwarder sits? It's impossible to modify this using props.conf?

The last resort should be to set the timezone in props.conf, because you are hard-coding values.

[richgalloway]

The TZ attribute should work on the UF, but isn't in your case so I offered solutions. You should open a case with Splunk Support to find out why it's not working as documented.

[dan_ce]

Good to know it should work, at least. Thanks for your help.

I've found a workaround which is to use _tzhint on the input stanza - works first time!