We've started noticing that some of the data from our Linux universal forwarders are being sent with a '.' appended to the indexed hostname. We've been running the 4.2.4 universal forwarder for a while now.
We're seeing a mixture of events from these hosts, some with the indexed hostname set to "host" and some with the name set to "host.". I've never seen this before.
I checked the hostname on each of the 4 servers we see this on, and none have the '.' appended. It's not in /etc/hosts either.
It's unclear where Splunk is getting this host value from, but in some cases it's clearly wrong. Has anyone seen this before?
Thanks
Curiously, we had a reason to reboot these servers later today. When we did this issue cleared up completely.
I did reverse lookups on all of these host IPs and they are all fully-qualified names in the correct form we'd expect. We are logging a tree full of different apache virtual servers -- all in the same way. That is, there's one set of rules for access.log and error.log in the whole tree. No other unique monitoring for Splunk on this server. Thanks.
I have seen this happen with DNS lookups on an internal windows DNS server before, is that similar to this setup and is it possible there is something logging on the box that is either having a DNS lookup done on its IP or perhaps is trying to lookup itself?