I recently upgraded all of my Universal Forwarders (UF) to 7.0.3 from various version levels (some 6.3.3, some were 7.x).
On one of the forwarders (AIX) when I run the command;
I get; "Splunk Universal Forwarder 7.0.3"
But a search to list forwarder versions on the indexer lists a different version for the same host;
index=_internal sourcetype=splunkd group=tcpin_connections | stats first(version) by hostname
I get; "Version 7.0.2"
Why are the versions being listed differently? After the upgrade, this is the only UF not listing 7.0.3 at the indexer.
Thanks for your reply. Same result I'm afraid. The Monitoring Console also reports version 7.0.2 for this host. I think that's how it's coming to the indexer. I just don't know why.
Pardon my negativity, it would not surprise me if this is a bug, surely the wrong version is being sent by the forwarder.
I would recommend uninstalling and reinstalling, if the same then obviously a defect!
Fixed! After backing up /opt/splunkforwarder/etc/auth/mycerts and /opt/splunkforwarder/etc/system/local I went ahead and got the Ops group to delete /opt/splunkforwarder/etc (and all subs). Uploaded a new copy of the install tar for 7.0.3 and had them do a gunzip to install.
On startup the server.pem file was missing for some reason so I grabbed a working copy from another 7.0.3 host and it worked.
The indexer is now reporting 7.0.3 for this forwarder properly!
This looks like it may be a bug in the AIX build. I just tried the linux tgz and Mac dmg, and they both look right.
I suggest opening a support case, it may have just been a bad build of the AIX UF.
Thanks for your comment ckurtz.
I have 6 AIX forwarders and all were updated at the same time. Only one of them is mis-reporting the version.
If I had access to do the install myself I would, but they have the servers so locked down here I have to open a change ticket which takes about 1/2 hour, wait for assessment by many people, and then wait a week to have the steps performed.