I need to set Windows forwarders to use the FQDN as the hostname across all inputs, as I have duplicate hostnames in my environment. I've tried changing everything in a Splunk deployment app, but the only thing that seems to work is if I manually correct the entry in C:\Program Files\SplunkUniversalForwarder\etc\system\local\inputs.conf from hostname to FQDN.
host = svr-vCenter
host = svr-vCenter.domain.local
The changes that I've made in the deployment app have meant that the splunkd.log reports the name correctly:
10-08-2016 16:46:59.063 +0100 INFO ServerConfig - Host name option is "fullyqualifiedname".
10-08-2016 16:46:59.063 +0100 INFO ServerConfig - My hostname is "svr-vCenter.domain.local".
But anything sent to the Splunk indexer shows up as the shortname and the FQDN gets ignored unless I change the file above.
Is there any way I can automatically set this during install or afterwards in Splunk other than manually changing the contents of the file? What am I missing as this is driving me nuts!
The funny thing is that when we deploy the apps via the deployment server, we specify the servers in the
serverclass.conf and we specify them using FQDN. So, why does the software determine the host name by itself after discovering it via set-up in
serverclass.conf? which leads to these very annoying discrepancies.
So, it would be nice if there was a way to carry the name from the
inputs.conf of the forwarder. It's true that sometimes, we specify a set of hosts in
serverclass.confusing wildcards, which might make the transition more complex.
Check server.conf in the same local folder. I think there is a hostname setting in there.
Regarding how to handle this across multiple installs, the Splunk forwarder will set the hostname at the time of install if you it is a fresh install. if you are copying system images that already contain the Splunk forwarder, you need to run an additional command to clear out the host-specific information. The command is
./splunk clone-prep-clear-config. If you are using this on a already cloned system, run that command and then restart the Splunk forwarder service. It should update its values upon restart.
Here are more detailed instructions about replicating Forwarder installation across multiple guests. http://docs.splunk.com/Documentation/Splunk/latest/Admin/Integrateauniversalforwarderontoasystemimag...
Thanks rjthibod, but I know that if I change the inputs.conf on each forwarder that it fixes the issue. If I have to do this manually it's going to take a long time to login to each and every server to make the change.
How can I accomplish this from the Splunk server without making changes directly on the forwarder?
Apologies for being slightly redundant.
As far as doing it from the Splunk server, the only thing I can think of that will change the files on the servers would be a custom deployment app.
From the Splunk Forwarder Management (Deployment Server), you would send an app to each forwarder. That app would just be a batch file or Powershell script that edits the file in the manner you require. I am pretty sure the app runs with the permissions allocated to the forwarder, so potential access control concerns issues could arise if the app doesn't run with system-level admin privileges needed to edit the configuration file.
Other than that, you could write a powershell script to perform remote commands on each server. That would be an exercise outside of Splunk (as you very well know).