I have a bit of problem with seeing the host name as FQDN. This is a problem both in the deployment server and the *NIX app. I noticed previously that the Deployment monitor wasn't seeing the FQDN, but that got fixed by this command on each forwarder:
splunk set servername $(hostname -f)
I noticed that this only changed
etc/system/local/server.conf and not
etc/system/local/inputs.conf. The change was visible in the Deployment Monitor, but not the Deployment Server or the *NIX app. So, I changed
[splunk@splunk-uf-01 splunkforwarder]$ find etc -name \*.conf |xargs grep $(hostname -s) etc/system/local/inputs.conf:host = splunk-uf-01.dom.ain.tld etc/system/local/server.conf:serverName = splunk-uf-01.dom.ain.tld
But this change is still not visible in the Deployment Server:
[splunk@splunk-dep-srv ~]$ splunk list deploy-clients |grep hostname: |grep uf-01 hostname: splunk-uf-01
This is a problem since some of my server classes rely on the FQDN. It is also annoying that the hostnames seen by the deployment server are mixed, ie. sometimes with the FQDN and sometimes without.
Any help would certainly be appreciated.
May be a silly point but did you restart the forwarder after making the changes? Also is the hostname for the box itself correct?
Some good debug to do could be to check there are no more instances of the incorrect hostname by running the following command on your UF's;
./splunk cmd btool server list --debug AND
./splunk cmd btool inputs list --debug
This will list all the config options from each of those config files, add a >> output.txt to the end so you can dump the output to a file and search that for any more occurances of the hostname. Its possible its defined elsewhere and overriding your current settings.
Here's what etc/system/local/inputs.conf looks like:
[default] host = indexer-11.dom.ain.tld [splunktcp-ssl:9996] [SSL] password = verysecret requireClientCert = true rootCA = /opt/splunk/etc/auth/cacert.pem serverCert = /opt/splunk/etc/auth/server.pem
Yes, I restarted the computer and printer before calling helpdesk. 😉 Joking aside, I restarted the universal forwarder and the the deployment server for good measure, but it didn't make a difference, I'm afraid...
The btool commands are useful and basically confirm that the FQDN is used throughout the configuration.
Any chance that the first version of the name is cached somewhere? I've only tried grepping *.conf and *.csv files on both the forwarder and deployment server.