Why does my KVStore replicationStatus show recovering?

satyaallaparthi — Mon, 13 Feb 2023 23:32:14 GMT

Hello,

I have a problem with my replication status.. getting the below result when ever I am trying to see kvstore status.

host 2 replication status is showing recovering from past 1 week.

I tried to resync both shcluster config using "splunk resync shcluster-replicated-config" and kvstore using "splunk resync kvstore -source MyGUID"

But, both didn't work.

Any help would be appreciated.

This member:
backupRestoreStatus : Ready
date : Mon Feb 3 15:50:19 2020
dateSec : 1580763019.583
disabled : 0
guid : ****-46ED-450B-BE21-******
oplogEndTimestamp : Mon Feb 3 15:50:18 2020
oplogEndTimestampSec : 1580763018
oplogStartTimestamp : Mon Feb 3 06:06:00 2020
oplogStartTimestampSec : 1580727960
port : 8191
replicaSet : splunkrs
replicationStatus : KV store captain
standalone : 0
status : ready

Enabled KV store members:
host1:8191
guid : ****-46ED-450B-BE21-******
hostAndPort : host1:8191
host2:8191
guid : ***-9A10-4962-9A3F-******
hostAndPort : host2:8191
host3:8191
guid : ****-F4B3-4E58-B765-*******
hostAndPort : host3:8191

KV store members:
host1:8191
configVersion : 205602
hostAndPort : host1:8191
lastHeartbeat : Mon Feb 3 15:50:17 2020
lastHeartbeatRecv : Mon Feb 3 15:50:17 2020
lastHeartbeatRecvSec : 1580763017.874
lastHeartbeatSec : 1580763017.873
optimeDate : Mon Feb 3 15:50:14 2020
optimeDateSec : 1580763014
pingMs : 0
replicationStatus : Non-captain KV store member
uptime : 253492

   host2:8191
                             configVersion : 205602
                               hostAndPort : host:8191
                             lastHeartbeat : Mon Feb  3 15:50:17 2020
                         lastHeartbeatRecv : Mon Feb  3 15:50:19 2020
                      lastHeartbeatRecvSec : 1580763019.366
                          lastHeartbeatSec : 1580763017.873
                                optimeDate : Wed Oct  2 11:09:46 2019
                             optimeDateSec : 1570028986
                                    pingMs : 0
                         replicationStatus : Recovering
                                    uptime : 253417

    host3:8191
                             configVersion : 205602
                              electionDate : Fri Jan 24 21:18:18 2020
                           electionDateSec : 1579918698
                               hostAndPort : host3:8191
                                optimeDate : Mon Feb  3 15:50:18 2020
                             optimeDateSec : 1580763018
                         replicationStatus : KV store captain
                                    uptime : 844587

Thanks,

Re: Need a help with my KVStore replicationStatus is recovering

jaihingorani — Sun, 31 May 2020 13:28:26 GMT

if resyn does not work, you can try below, it has been working for me so far :
1. stop the splunk.
2../splunk clean kvstore --local
3. start splunk.

This triggers the initial Synchronization from other KVstore members

Re: Need a help with my KVStore replicationStatus is recovering

humrish_b — Fri, 15 Oct 2021 11:42:10 GMT

@satyaallaparthi did you try the steps provided by @jaihingorani was it successful. Could you let me know how this issue is resolved.

Re: Need a help with my KVStore replicationStatus is recovering

mustapha_arakji — Mon, 13 Feb 2023 22:17:10 GMT

Your answer is in here: https://docs.splunk.com/Documentation/Splunk/9.0.3/Admin/ResyncKVstore

topic Re: Need a help with my KVStore replicationStatus is recovering in Knowledge Management

Why does my KVStore replicationStatus show recovering?

Re: Need a help with my KVStore replicationStatus is recovering

Re: Need a help with my KVStore replicationStatus is recovering

Re: Need a help with my KVStore replicationStatus is recovering