Some log events do not have timezone information in it so I need to set the timezone in the props.conf on the forwarder. This works fine however we have many universal forwarders in multiple timezone and it would be useful to have one standard build. Is it possible to get the forwarder to get the timezone information from the underlying OS?
As of version 6, Splunk forwarders provide the local OS timezone as the default. If the data (the log file or whatever) does not specify a timezone, the local OS timezone will be used.
"If an event that arrives at an indexer originated at a forwarder, and both the forwarder and the receiving indexer run Splunk Enterprise 6.0 or later, then Splunk Enterprise uses the time zone that the forwarder provides." from the Getting Data In manual.
I saw that in the documentation but I don't think it works that way. If there is no timezone information in the event and nothing in the props.conf on the forwarder then there will be no timezone information sent from the forwarder. If there was then the last statement in the documentation would be redundant "Splunk Enterprise uses the time zone of the server that indexes the event. ". Or I am misreading the documentation?
I assure that it works that way. As of Splunk 6.0, the data packet sent from the forwarder to the indexer always includes basic info about the forwarder itself, including the forwarder's local system timezone.
You are misreading the documentation. "Splunk Enterprise uses the time zone of the server that indexes the event." means that, if all else fails, Splunk will use the indexer's timezone.
It is very common for Splunk forwarders to be versions behind the indexers. So if you have a 5.x forwarder, you can certainly forward to a 6.x indexer. In that case, there will be no forwarder local system time - and the default timezone will be the timezone of the indexer. If you have a 6.x forwarder, the default will be the timezone of the forwarder.
I am getting very odd behaviour here and it is more complex that I originally thought. What I am going to do is open a case with Splunk support - even though I believe you work for Splunk, I need this tracked. Thanks for reaching out to me and trying to fix it. Tony