topic Re: Why are multiple host names being reported for the same host? in Getting Data In

Why are multiple host names being reported for the same host?

Michael — Tue, 29 Sep 2020 11:59:48 GMT

'Morning...
I have a v6.5, clustered environment (deployment server), Universal Forwarder on all hosts.

I am getting several Linux systems reporting in with two names, shortname and FQDN. But not all of them are doing this, even members of the same Server Class.

It seems that all the shortnames are only pulling a sourcetype of syslog or linux_messages_syslog and are only source=/var/log/messages.

The FQDNs are showing appropriate sourcetypes and sources (all under /var/log/ -- but NOT messages).

I have a very simple inputs.conf being deployed:

[monitor:///var/log]
index = servers
disabled = 0

I confirmed that syslog is not configured on these to also send to my heavy forwarders. They are reporting in to the Forwarder Management interface as one system (mixture of short and FQDN).

I haven't found a lot of mentions of this here -- I guess this is not very common...?

Thoughts?
Thanks!
Michael

Re: Why are multiple host names being reported for the same host?

gcusello — Tue, 29 Sep 2020 11:59:50 GMT

Hi Michael,
If you have only forwarders and none syslog you have to verify the servername associated to the Splunk Forwarder.
You can verify it in your servers in $SPLUNK_HOME/etc/system/local/server.conf and in $SPLUNK_HOME/etc/system/local/inputs.conf.
Servername is associated at the installation time from the server hostname.
If you want, you can modify it but in both the conf files.
Bye.
Giuseppe

Re: Why are multiple host names being reported for the same host?

Michael — Tue, 29 Nov 2016 18:51:41 GMT

Yes, I know where to find the hostname, but thanks.

For what it's worth, I've confirmed that both the inputs.conf and server.conf have the FQDN of the system.

Even /etc/hostname on the system has the FQDN.

Re: Why are multiple host names being reported for the same host?

Michael — Mon, 13 Mar 2017 14:43:13 GMT

FYI, trying $decideonStartUp didn't seem to work. To recap:

[monitor:///var/log]
host = $decideOnStartup
index = atl
disabled = 0

Mix of FQDN and short names sending in these:
/var/log/audit/audit.log
/var/log/cron

/var/log/mcelog
/var/log/up2date
/var/log/rhsm/rhsm.log
etc...

short only no FQDN
/var/log/messages

Re: Why are multiple host names being reported for the same host?

DalJeanis — Mon, 13 Mar 2017 15:15:07 GMT

Are some of the logs duplicated, or is it either/or?

If either/or, then can you post sanitized versions of each kind?

Re: Why are multiple host names being reported for the same host?

Michael — Mon, 13 Mar 2017 15:40:36 GMT

None of them are duplicated.

Of the three specific ones (in this section of my organization) that I'm narrowing down on, they send everything fine from /var/log/ using their FQDN -- but only the /var/log/messages file is reported using the short-name. They have other Linux (RHEL and CentOs) in that area that are reporting in using shortnames only. I'm trying to find out how they're different...

Ah, I see why you wanted a sample of the logs -- the /var/log/message file does include the hostname (short) -- seems that Splunk is pulling the name from there.(?) In the other log files, it does not included a name -- so it's getting it from DNS (hence, the FQDN).

i.e.:
Mar 13 06:18:29 servername dhclient[2958]: DHCPACK from 10.14.8.82 (xid=0x369db4ff)
v.s.
type=CRED_DISP msg=audit(1489419002.019:169929): user pid=26065 uid=0 auid=0 ses=25846 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'

Is that where you were going with that? I still see other hosts in the same area that are reporting with the same syntax -- but they're not producing duplicate names.

OK, now, how to fix that?

(I love it when I'm apparently the only one "out there" that's experienced are particular issue... 😉

Cheers,

Re: Why are multiple host names being reported for the same host?

ridwanahmed — Thu, 19 Sep 2019 14:53:39 GMT

Did you ever find a solution to this? I'm having a similar issue-- only source=var/log/messages goes to host=hostname, while the other logs have host=fqdn

my inputs.conf--the only place where this host name is defined (it's not in server.conf)

[default]
host = myhost.mydomain.com
[monitor:///var/log/messages]
disabled = false
sourcetype = syslog
index = myindex_1

[monitor:///usr/local/tomcat/logs/logname.log]
disabled = false
sourcetype = log4j i
index = myindex_2

Thanks for any comments.

Re: Why are multiple host names being reported for the same host?

ridwanahmed — Thu, 19 Sep 2019 15:57:41 GMT

For someone else with this issue-- the sourcetype of "syslog" has a specific transform set up to pull out the hostname from the logs. It doesn't use the inputs.conf or server.conf name

See this answer:
https://answers.splunk.com/answers/55751/host-field-getting-overwritten-in-syslog-processing.html