How can I use the fully qualified domain name (FQDN) as the host value for /var/log/messages?

[MillerTime]

With a universal forwarder installed on a Linux host (many flavors including Ubuntu & CentOS), the Splunk indexer will index events with two different hosts values for a single host: the host value (first part) and the fully qualified domain name (FQDN).

Example: /var/log/messages has host value set to 'webdev01' but other log files and metrics show the host value set to 'webdev01.dev.splunk.com'. I want all events from this host to be set to the FQDN (webdev01.dev.splunk.com).

[TonyLeeVT]

It would be nice if the Splunk UF had a way to change the host variable from hostname to fqdn or something similar.

If you have the following scenario it will cause issues:
host1.red.net
host1.blue.net

Both hosts show up as host=host1 which is obviously not ideal.

[rbal_splunk]

1) Please not that Splunk use socket and its gethostname() functionality to get the hostname of the computer where the Python interpreter is running. In your case you could run the folloing command to check it out

---

$python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import socket
socket.gethostname()

***********

The above command will list the host name – when splunk is installed and started for the 1st time it use this function and set the value of the host name in files

$SPLUNk_HOME/etc/system/local/inputs.conf

[default]
host =

$SPLUNk_HOME/etc/system/local/server.conf

[general]
serverName =

In your case you have two options:

1)Assuming this change is only for forwadwer

1.2) Change the following attribute to FQDN

$SPLUNk_HOME/etc/system/local/inputs.conf
[default]
host =

$SPLUNk_HOME/etc/system/local/server.conf
[general]
serverName =

When you make above change it will only change the host name for the new incoming data – and it won’t impact the already indexed data, so you might have to use the search query to accommodating both host name. Also, this suggestion is relevant only for Forwarder, on indexer and Search Head this will be more planned and tested approach.

1.3) The other option will be to address this outside of system and by OS administrator. Here you will need to make OS level configuration such that the python “socket.gethostname()” redurin FQDN

$python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import socket
socket.gethostname()

***********

[MarioM]

And did your try the following in syslog-ng.conf:

options {

         use_fqdn(yes);

[hexx]

I suspect that you could attain a similar result by suppressing the "syslog-host" index-time field extraction that is applied to the "syslog" sourcetype by default. This would prevent splunkd from re-writing the "host" field with the value extracted from _raw and the value of "host" set in inputs.conf (which is usually the FQDN) will be retained.

[MillerTime · Accepted solution]

There are multiple ways to make sure the host value is set to the fully qualified domain name using Splunk configuration files (props/transforms to change host value at index-time, for example), but the answer below is an OS configuration solution. The reason for this is that by default the first part of the hostname is used as the value in /var/log/messages, not the FQDN. The solution below doesn't require a host-specific configuration in Splunk.

Note: this solution requires the Linux host to use rsyslog and not syslog. You can check this by verifying the existence of following file: /etc/rsyslog.conf. If you're not using rsyslog then you're out of luck for this solution. If this is the case you'll want to replace syslog with rsyslog as your default kernel logger, otherwise you'll need to use host-specific configs on the forwarders, or a props/transforms rule at the indexer to modify the host value.

There are three files that you need to modify to ensure that Splunk uses the fully qualified domain name for all of the events' host value:

1. /etc/hosts
   --example: 192.168.0.5 webdev01.dev.splunk.com webdev01

2. /etc/sysconfig/network
   --example: HOSTNAME=webdev01.dev.splunk.com

3. /etc/rsyslog.conf
   -- Under #### GLOBAL DIRECTIVES ####
   -- add the line: $PreserveFQDN on
   -- (restart rsyslog)