The Monitoring Console uses metrics data provided by servers with a splunk forwarder installed. The metrics data appears to use the hostname found on linux servers in the /etc/hostname file. However, our forwarders are set up with a hostname specified in the ../etc/system/local/inputs.conf where a "cname" for the host is specified. This results in a difference between the "host" used in searches and the "hostname" specified in the Monitoring Console dashboards and alerts. Is there a best practice for unifying the host and hostname in the Monitoring Console?
The value used for the host in the metrics.log which I believe is the logs you are referring to which powers some of the Monitoring Console dashboards comes from the "serverName" field under the [general] stanza of server.conf
If you update your /opt/splunk/etc/system/local/server.conf file so that the serverName value under [general] is the correct name for your host then this should flow through to the Monitoring Console.
Let me know how you get on!
Regards
Will
Thanks for the reply. What you said makes sense. I have a concern though. I looked at one of our typical UF installs and I verified that there already is a ../etc/system/local/server.conf. Since I'm the admin and normally do all UF deployments, I know that this file was automatically generated when the forwarder was installed. As you suspected, it contains the hostname of the server. Interestingly, the ../etc/system/default/server.conf contains serverName = $HOSTNAME. So the serverName field is populated when the UF is installed and a local/server.conf is created. The issue I have is that this would have to be overridden after a UF is installed. This is possible, but seems like it shouldn't be necessary. Thoughts?