https://docs.splunk.com/Documentation/Splunk/9.0.0/Admin/MigrateKVstore#Upgrade_KV_store_server_to_version_4.2 Upgraded Splunk Enterprise version 9.0.0 from 8.2.5
Looking to see how to upgrade mongo from 4.0 to 4.2 on a single instance deployment. During the Splunk Enterprise upgrade the migration to wiredTiger failed due to lack of disk space, the upgrade still continued and made the first hop of the mongo upgrade from version 3.6 to 4.0, it looks like after version 4.0 it tried to do the engine migration but couldn't because the lack of available disk space and therefore didn't do the last hop to version 4.2 of mongo. We have since fixed the disk space issue and were able to complete the engine migration to wiredTiger, however don't know how to bump up the mongo version to 4.2. The above link is for upgrading mongo in a cluster but not on a single instance, when looking at the options in splunk help kvstore I don't see anything for upgrading mongo either for a single instance, tried splunk start-shcluster-upgrade kvstore -version 4.2 -isDryRun true but of course it detected it wasn't a searchhead cluster. Lastly trying to understand the difference in the output of mongo versionsbetween kvstore-status command versus splunk cmd mongod -version, clearly pulling from two different places. [App Key Value Store migration] Starting migrate-kvstore. Created version file path=/opt/splunk/var/run/splunk/kvstore_upgrade/versionFile36 Started standalone KVStore update, start_time="2022-06-22 15:21:46". [App Key Value Store migration] Checking if migration is needed. Upgrade type 1. This can take up to 600seconds. [App Key Value Store migration] Migration is not required. Created version file path=/opt/splunk/var/run/splunk/kvstore_upgrade/versionFile40 Not enough space to upgrade KVStore (or backup). You will need requiredBytes=3107201024 bytes, but KV Store DB filesystem only has availableBytes=2286272512 [App Key Value Store migration] Starting migrate-kvstore. [App Key Value Store migration] Storage Engine hasn't been migrated to wireTiger. Cannot upgrade to service(42)
[splunk ~/var/run/splunk/kvstore_upgrade]$ splunk show kvstore-status --verbose |grep serverVersion serverVersion : 4.0.24 [splunk ~/var/run/splunk/kvstore_upgrade]$
[splunk ~/var/run/splunk/kvstore_upgrade]$ splunk cmd mongod -version db version v4.2.17-linux-splunk-v3 git version: be089838c55d33b6f6039c4219896ee4a3cd704f OpenSSL version: OpenSSL 1.0.2zd-fips 15 Mar 2022 allocator: tcmalloc modules: none build environment: distmod: rhel62 distarch: x86_64 target_arch: x86_64 [splunk ~/var/run/splunk/kvstore_upgrade]$
... View more