We have logger_cef data that is processed by our heavy forwarder. The host value in the event is actually the Splunk forwarder name. Actual host value is called dvchost. Therefore, I can not use host field to offset the TZ in props.
I'm trying to figure out if there is a way to use props/transforms to rewrite the dvchost value to host, and THEN offset based on new host value.
Yes, props/transforms can rewrite host (or source, sourcetype or even index) at parsing time. In your instance, this would have to go on the heavy forwarder, since it's parsing the events. The basic gist is below:
# This is the default, but it never hurts to be specific about where you're looking
SOURCE_KEY = _raw
# Look for this, the thing you want (the value of dvchost) should be in a capture group
# What to set
DEST_KEY = MetaData:Host
FORMAT = host::$1
Look at the docs for transforms.conf for more info about the special MetaData keys.
But you probably know all of that. Now the trick is how to use this new host to set the TZ. Unfortunately, that doesn't seem to be possible. The reference doc I have for the Splunk parsing pipeline shows that all of the time-related things TIME_PREFIX, TIME_FORMAT, TZ, etc are all processed before any named parse-time TRANSFORMS. However, that particular documentation entry indicates that the DEST_KEY of a parse-time TRANSFORM can include _time. So in theory, you could write a regex keying off of the new hostname (with SOURCE_KEY and REGEX set appropriately) and then set FORMAT to something relative, but... that presumes you'd have the epoch time in a capture group somewhere, and that seems unlikely since you're only looking in the MetaData:host field.
So maybe have the HF that's receiving these events try to get the host right in the first place? Maybe consider some of the options on inputs.conf, like connection_host to see if it can have the right hostname when the time comes to apply host-based props for time zone offset?
Another option would be to re-parse the data once it arrive at the indexer by changing the queue field in the indexer's inputs.conf, now with the correct hostname stored in the host field to key off of for the time zone offset.
Doesn't sound easy. Hopefully some of this was helpful.