Installation

How to upgrade Mongo in Splunk 9.0.0?

amartin6
Path Finder

https://docs.splunk.com/Documentation/Splunk/9.0.0/Admin/MigrateKVstore#Upgrade_KV_store_server_to_v...

Upgraded Splunk Enterprise version 9.0.0 from 8.2.5

Looking to see how to upgrade mongo from 4.0 to 4.2 on a single instance deployment.  During the Splunk Enterprise upgrade the migration to wiredTiger failed due to lack of disk space, the upgrade still continued and made the first hop of the mongo upgrade from version 3.6 to 4.0, it looks like after version 4.0 it tried to do the engine migration but couldn't because the lack of available disk space and therefore didn't do the last hop to version 4.2 of mongo.  We have since fixed the disk space issue and were able to complete the engine migration to wiredTiger, however don't know how to bump up the mongo version to 4.2.  The above link is for upgrading mongo in a cluster but not on a single instance, when looking at the options in splunk help kvstore I don't see anything for upgrading mongo either for a single instance, tried splunk start-shcluster-upgrade kvstore -version 4.2 -isDryRun true but of course it detected it wasn't a searchhead cluster.  Lastly trying to understand the difference in the output of mongo versionsbetween kvstore-status command versus splunk cmd mongod -version, clearly pulling from two different places.

[App Key Value Store migration] Starting migrate-kvstore.
Created version file path=/opt/splunk/var/run/splunk/kvstore_upgrade/versionFile36
Started standalone KVStore update, start_time="2022-06-22 15:21:46".
[App Key Value Store migration] Checking if migration is needed. Upgrade type 1. This can take up to 600seconds.
[App Key Value Store migration] Migration is not required.
Created version file path=/opt/splunk/var/run/splunk/kvstore_upgrade/versionFile40
Not enough space to upgrade KVStore (or backup). You will need requiredBytes=3107201024 bytes, but KV Store DB filesystem only has availableBytes=2286272512
[App Key Value Store migration] Starting migrate-kvstore.
[App Key Value Store migration] Storage Engine hasn't been migrated to wireTiger. Cannot upgrade to service(42)

[splunk ~/var/run/splunk/kvstore_upgrade]$ splunk show kvstore-status --verbose |grep serverVersion
serverVersion : 4.0.24
[splunk ~/var/run/splunk/kvstore_upgrade]$

[splunk ~/var/run/splunk/kvstore_upgrade]$ splunk cmd mongod -version
db version v4.2.17-linux-splunk-v3
git version: be089838c55d33b6f6039c4219896ee4a3cd704f
OpenSSL version: OpenSSL 1.0.2zd-fips 15 Mar 2022
allocator: tcmalloc
modules: none
build environment:
distmod: rhel62
distarch: x86_64
target_arch: x86_64
[splunk ~/var/run/splunk/kvstore_upgrade]$

Labels (1)
Tags (3)
1 Solution

amartin6
Path Finder

We were able to upgrade mongo via the migrate command, thought it was only for the engine:

[splunk@ ~]$ splunk migrate migrate-kvstore
[App Key Value Store migration] Starting migrate-kvstore.
[App Key Value Store migration] Checking if migration is needed. Upgrade type 2. This can take up to 600seconds.
[App Key Value Store migration] Migration is not required.
Created version file path=/opt/splunk/var/run/splunk/kvstore_upgrade/versionFile42
Finished standalone KVStore update, stop_time="2022-06-29 13:41:29".
[splunk@ ~]$

[splunk@ ~]$ splunk show kvstore-status --verbose |grep serverVersion
serverVersion : 4.2.17
[splunk@ ~]$

View solution in original post

amartin6
Path Finder

We were able to upgrade mongo via the migrate command, thought it was only for the engine:

[splunk@ ~]$ splunk migrate migrate-kvstore
[App Key Value Store migration] Starting migrate-kvstore.
[App Key Value Store migration] Checking if migration is needed. Upgrade type 2. This can take up to 600seconds.
[App Key Value Store migration] Migration is not required.
Created version file path=/opt/splunk/var/run/splunk/kvstore_upgrade/versionFile42
Finished standalone KVStore update, stop_time="2022-06-29 13:41:29".
[splunk@ ~]$

[splunk@ ~]$ splunk show kvstore-status --verbose |grep serverVersion
serverVersion : 4.2.17
[splunk@ ~]$

richgalloway
SplunkTrust
SplunkTrust

You may want to contact support for assistance.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...