Hi , I tried to upgrade splunk universal forwarder from 7.0.2 to 8.0.2 and everything looks good ,
No error in splunkd logs
Data is ingesting normally and all internal logs are also coming fine.
But when i look into migration.log and i saw these messages , is it be any problem ;
[App Key Value Store migration] Binary for service(34) is missing.
As far as i know this related to KV store migartion and splunk forwarder wont use it, Please if any one can help on this ?
There is no kvstore on a universal forwarder, so it's not an issue. i just installed a v7.0.2 forwarder on my host and upgraded it to 8.0.2 and got the same error... you're good to ignore it.
darren@Universe11-026:~/Downloads$ sudo tar -xzf ./splunkforwarder-7.0.2-03bbabbd5c0f-Linux-x86_64.tgz -C /opt
[sudo] password for darren:
darren@Universe11-026:~/Downloads$ cd /opt/splunkforwarder/bin
darren@Universe11-026:/opt/splunkforwarder/bin$ ./splunk start --accept-license --answer-yes
Splunk> Take the sh out of IT.
Checking prerequisites...
Checking mgmt port [8089]: open
Creating: /opt/splunkforwarder/var/lib/splunk
Creating: /opt/splunkforwarder/var/run/splunk
Creating: /opt/splunkforwarder/var/run/splunk/appserver/i18n
Creating: /opt/splunkforwarder/var/run/splunk/appserver/modules/static/css
Creating: /opt/splunkforwarder/var/run/splunk/upload
Creating: /opt/splunkforwarder/var/spool/splunk
Creating: /opt/splunkforwarder/var/spool/dirmoncache
Creating: /opt/splunkforwarder/var/lib/splunk/authDb
Creating: /opt/splunkforwarder/var/lib/splunk/hashDb
New certs have been generated in '/opt/splunkforwarder/etc/auth'.
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from '/opt/splunkforwarder/splunkforwarder-7.0.2-03bbabbd5c0f-linux-2.6-x86_64-manifest'
All installed files intact.
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Done
darren@Universe11-026:/opt/splunkforwarder/bin$ tar -xzf /home/darren/Downloads/splunkforwarder-8.0.2.1-f002026bad55-Linux-x86_64.tgz -C /opt
darren@Universe11-026:/opt/splunkforwarder/bin$ ./splunk restart --accept-license --answer-yes
Stopping splunkd...
Shutting down. Please wait, as this may take a few minutes.
Stopping splunk helpers...
Done.
This appears to be an upgrade of Splunk.
--------------------------------------------------------------------------------)
Splunk has detected an older version of Splunk installed on this machine. To
finish upgrading to the new version, Splunk's installer will automatically
update and alter your current configuration files. Deprecated configuration
files will be renamed with a .deprecated extension.
You can choose to preview the changes that will be made to your configuration
files before proceeding with the migration and upgrade:
If you want to migrate and upgrade without previewing the changes that will be
made to your existing configuration files, choose 'y'.
If you want to see what changes will be made before you proceed with the
upgrade, choose 'n'.
Perform migration and upgrade without previewing configuration changes? [y/n] y
-- Migration information is being logged to '/opt/splunkforwarder/var/log/splunk/migration.log.2020-03-25.16-49-01' --
Migrating to:
VERSION=8.0.2.1
BUILD=f002026bad55
PRODUCT=splunk
PLATFORM=Linux-x86_64
It seems that the Splunk default certificates are being used. If certificate validation is turned on using the default certificates (not-recommended), this may result in loss of communication in mixed-version Splunk environments after upgrade.
"/opt/splunkforwarder/etc/auth/ca.pem": already a renewed Splunk certificate: skipping renewal
"/opt/splunkforwarder/etc/auth/cacert.pem": already a renewed Splunk certificate: skipping renewal
[App Key Value Store migration] Binary for service(34) is missing.
[App Key Value Store migration] Binary for service(34) is missing.
[DFS] Performing migration.
[DFS] Finished migration.
Splunk> Take the sh out of IT.
Checking prerequisites...
Checking mgmt port [8089]: open
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from '/opt/splunkforwarder/splunkforwarder-8.0.2.1-f002026bad55-linux-2.6-x86_64-manifest'
All installed files intact.
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Done
There is no kvstore on a universal forwarder, so it's not an issue. i just installed a v7.0.2 forwarder on my host and upgraded it to 8.0.2 and got the same error... you're good to ignore it.
darren@Universe11-026:~/Downloads$ sudo tar -xzf ./splunkforwarder-7.0.2-03bbabbd5c0f-Linux-x86_64.tgz -C /opt
[sudo] password for darren:
darren@Universe11-026:~/Downloads$ cd /opt/splunkforwarder/bin
darren@Universe11-026:/opt/splunkforwarder/bin$ ./splunk start --accept-license --answer-yes
Splunk> Take the sh out of IT.
Checking prerequisites...
Checking mgmt port [8089]: open
Creating: /opt/splunkforwarder/var/lib/splunk
Creating: /opt/splunkforwarder/var/run/splunk
Creating: /opt/splunkforwarder/var/run/splunk/appserver/i18n
Creating: /opt/splunkforwarder/var/run/splunk/appserver/modules/static/css
Creating: /opt/splunkforwarder/var/run/splunk/upload
Creating: /opt/splunkforwarder/var/spool/splunk
Creating: /opt/splunkforwarder/var/spool/dirmoncache
Creating: /opt/splunkforwarder/var/lib/splunk/authDb
Creating: /opt/splunkforwarder/var/lib/splunk/hashDb
New certs have been generated in '/opt/splunkforwarder/etc/auth'.
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from '/opt/splunkforwarder/splunkforwarder-7.0.2-03bbabbd5c0f-linux-2.6-x86_64-manifest'
All installed files intact.
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Done
darren@Universe11-026:/opt/splunkforwarder/bin$ tar -xzf /home/darren/Downloads/splunkforwarder-8.0.2.1-f002026bad55-Linux-x86_64.tgz -C /opt
darren@Universe11-026:/opt/splunkforwarder/bin$ ./splunk restart --accept-license --answer-yes
Stopping splunkd...
Shutting down. Please wait, as this may take a few minutes.
Stopping splunk helpers...
Done.
This appears to be an upgrade of Splunk.
--------------------------------------------------------------------------------)
Splunk has detected an older version of Splunk installed on this machine. To
finish upgrading to the new version, Splunk's installer will automatically
update and alter your current configuration files. Deprecated configuration
files will be renamed with a .deprecated extension.
You can choose to preview the changes that will be made to your configuration
files before proceeding with the migration and upgrade:
If you want to migrate and upgrade without previewing the changes that will be
made to your existing configuration files, choose 'y'.
If you want to see what changes will be made before you proceed with the
upgrade, choose 'n'.
Perform migration and upgrade without previewing configuration changes? [y/n] y
-- Migration information is being logged to '/opt/splunkforwarder/var/log/splunk/migration.log.2020-03-25.16-49-01' --
Migrating to:
VERSION=8.0.2.1
BUILD=f002026bad55
PRODUCT=splunk
PLATFORM=Linux-x86_64
It seems that the Splunk default certificates are being used. If certificate validation is turned on using the default certificates (not-recommended), this may result in loss of communication in mixed-version Splunk environments after upgrade.
"/opt/splunkforwarder/etc/auth/ca.pem": already a renewed Splunk certificate: skipping renewal
"/opt/splunkforwarder/etc/auth/cacert.pem": already a renewed Splunk certificate: skipping renewal
[App Key Value Store migration] Binary for service(34) is missing.
[App Key Value Store migration] Binary for service(34) is missing.
[DFS] Performing migration.
[DFS] Finished migration.
Splunk> Take the sh out of IT.
Checking prerequisites...
Checking mgmt port [8089]: open
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from '/opt/splunkforwarder/splunkforwarder-8.0.2.1-f002026bad55-linux-2.6-x86_64-manifest'
All installed files intact.
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Done
Thank you so much