universal forwarder mishandles serverName

[arthurhamm]

I am running Splunk Server and Universal Forwarder 4.2.1 98164. The config file "/opt/splunkforwarder/etc/system/local/server.conf" has the entry "serverName = nascpmpa1dr". This seems to work as the results of "/opt/splunkforwarder/bin/splunk show servername" give the proper result, "Server name: nascpmpa1dr". But my Indexer sees the server as "nascpmpa1", which in what my linux servers $HOSTNAME is set to. DNS resolves "nascpmpa1dr". I have this setup with several linux servers using Splunk Light Forwarder 4.1 and they all give the hostname with the DR appended. Why does the Indexer file the syslog and warn logs under host="nascpmpa1" and not "nascpmpa1dr"? Why act differently between SLF 4.1 and UF 4.2.1?

[gkanapathy]

The entry in server.conf is used only for identifying indexers when Splunk distributed searches is used. It has nothing to do with how data is marked with a host name when it is indexed. (It is used to populate the splunk_server field in results, but this is added at search time by the distributed indexer returning results.)

It has no relationship or effect on forwarding or indexing of data. For that you need to look at the host setting for an input in inputs.conf. If this is unspecified for an input, then 4.2.x uses the output of the hostname command. If unspecified, then 4.1.x and down uses the IP address, but 4.1.x sets a local default on first-time run to the results of the hostname command at the time of first-time run. You can use btool to see if host is set for a particular input.

---

Update: Additionally, the default value for serverName in server.conf (remember, it is not relevant except for distributed search internal to Splunk) uses the value of either $HOSTNAME or $HOSTNAME-$USER depending on version, which may not be the same as the results of hostname.

[kristian_kolb]

Thanks! No, having duplicate GUIDs could be a ton of hassle. Been down that road... Just wanted to be sure that the lack of a serverName entry would not cause unforseen issues. Thanks again.

/K

[gkanapathy]

provided your pre-made file doesn't contain the guid or serverName entries, it should be fine. Splunk will generate a new guid for the forwarder if one is missing (i suppose you could live with all of them having the same guid, but it may cause reporting and other problems.

[kristian_kolb]

Would you say it's safe to delete/replace the /etc/system/local/server.conf right after installing UF (before it's started for the first time)?

The reason is that we want to set some SSL configuration for connecting to the deployment server, and it seems easy to just drop in a pre-made server.conf (which naturally does not contain the serverName at all).

Thanks in advance,

Kristian

[ftk]

The universal forwarder does behave differently in 4.2.0 and 4.2.1 than a Light/Heavy Forwarder did in 4.1.x (SPL-38141, check the Known Issues). Work is under way to resolve this issue.

[ftk]

That's funny as it is listed under the known issues (data inputs) as well. No idea which one is correct.

[arthurhamm]

Universal Forwarder 4.2.1 98164 release notes lists SPL-38141 as a resolved issue.

http://www.splunk.com/base/Documentation/4.2.1/ReleaseNotes/4.2.1

[ftk]

The title of the bug does not reflect every facet of the issue.

[arthurhamm]

All my hostnames are in lowercase. And the clipping of the "dr" off the names makes me think it not this bug.