We have a setup where we have a syslog-ng server that forwards all events using a UF to a HF and then to the cloud. The issue we are having is that the host information is getting replaced with that of the UF name not the actual host that sent the syslog.
I don't have anything in the outputs.conf or inputs.conf on the UF setting the host. If I send directly to Splunk Cloud it will keep the correct host name. It is only when I send to the HF will this name get stripped and the host gets changed to the syslog server's name.
I have tried a regex to dynamically assign the host name in the inputs.conf by way of a regex based on the file path name on the UF, but cannot get it to work. An example of the file path is /var/log/splunk/network/hostname_log. I need just the hostname to be come the host.
My thought is that there must be a setting somewhere either on the UF or the HF that is doing this. Any ideas or is there another way of doing the.
My guess, without more information, is this: The universal forwarder collects the information and sends it to the heavy forwarder. The heavy forwarder parses the data; since no value is set for the host, it applies the uses the name of the forwarder as the host. Then the data is sent onward, already parsed, to the cloud indexers, which do no further processing but write the data to the index.
When the universal forwarder sends the information directly to cloud, the cloud indexers parse the data.
It is most likely that the parsing rules are set differently on the cloud indexers and the heavy forwarder. This could be because apps are installed in one location but not the other. The parsing is usually based on the sourcetype specified on the forwarder in inputs.conf or props.conf.