KVStore Migration to WiredTiger: Cluster Upgrade 8.0 > 8.2

Path Finder

We are planning upgrade our clustered deployment (SH-C + IDX-C) from 8.0.5 to 8.2.1

In case we plan to migrate the KVStore engine to WiredTiger, do we have to mandatorily go through the v8.1 step as mentioned in ? 

Per my understanding, the steps for this method stands like this

  1. Upgrade cluster from 8.0.5 to 8.1.5 
  2. Migrate the KV store after an upgrade to Splunk Enterprise 8.1 in a clustered deployment
  3. Upgrade cluster from 8.1.5 to 8.2.1

In case we decide not to use WiredTiger for now and complete the upgrade to 8.2.1, can we migrate the KV  Store to WiredTiger at a later point of time? 
To elaborate, will the following work: 

  1. Upgrade cluster from 8.0.5 to 8.2.1 
  2. At a later time, migrate the KV Store as instructed in Migrate the KV store after an upgrade to Splunk Enterprise 8.1 in a clustered deployment

We do not use KV Store at all now, apart from whatever internal functions that Splunk Enterprise uses it for (no ITSI or ES as well), but want to plan ahead in case we use it in the future. 

Thanks in advance!

Labels (2)
0 Karma

Path Finder

I just tested this on a standalone instance

  1. Splunk Enterprise v8.0.5  
  2. Set 
    [kvstore] storageEngineMigration=true
  3. Upgrade to v8.2.1
  4. Success! 


/opt/splunk/var/log/splunk/migration.log.2021-07-30: (only the relevant portion regarding kvstore migration)
Starting KV Store storage engine upgrade:
Phase 1 (dump) of 2:
Phase 2 (restore) of 2:
...................Data sanity check completed successfully for all configured KV Store collections
Successfully migrated to storage engine wiredTiger

(dev2) splunk@splnkhfvm-3hqdt:~ $ ./bin/splunk btool --debug server list kvstore | grep local
/opt/splunk/etc/system/local/server.conf   [kvstore]
/opt/splunk/etc/system/local/server.conf   storageEngine = wiredTiger
/opt/splunk/etc/system/local/server.conf   storageEngineMigration = true

(dev2) splunk@splnkhfvm-3hqdt:~ $ ./bin/splunk show kvstore-status

 This member:
                           backupRestoreStatus : Ready
                                          date : Fri Jul 30 12:19:36 2021
                                       dateSec : 1627661976.923
                                      disabled : 0
                                          guid : 8297E92E-9E18-40DB-865C-54F7D9F82CBF
                             oplogEndTimestamp : Fri Jul 30 12:19:29 2021
                          oplogEndTimestampSec : 1627661969
                           oplogStartTimestamp : Fri Jul 30 11:35:07 2021
                        oplogStartTimestampSec : 1627659307
                                          port : 8191
                                    replicaSet : 8297E92E-9E18-40DB-865C-54F7D9F82CBF
                             replicationStatus : KV store captain
                                    standalone : 1
                                        status : ready
                                 storageEngine : wiredTiger

 KV store members:
                                 configVersion : 1
                                  electionDate : Fri Jul 30 11:35:07 2021
                               electionDateSec : 1627659307
                                   hostAndPort :
                                    optimeDate : Fri Jul 30 12:19:29 2021
                                 optimeDateSec : 1627661969
                             replicationStatus : KV store captain
                                        uptime : 2675


So kvstore migration was a success when upgrading Splunk Enterprise from 8.0 to 8.2 in standalone mode. 

Next step: Validate if this is true with SH Cluster upgrade as well. 

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 2 releases of new security content via the ...

Announcing the 1st Round Champion’s Tribute Winners of the Great Resilience Quest

We are happy to announce the 20 lucky questers who are selected to be the first round of Champion's Tribute ...

We’ve Got Education Validation!

Are you feeling it? All the career-boosting benefits of up-skilling with Splunk? It’s not just a feeling, it's ...