Knowledge Management

How to resolve error on a search head member in the search head cluster: "Local KV Store has replication issues...Local instance has state Recovering."?

kranitha
Explorer

Hi All,

One of the search head members in the search head cluster has a message: "Local KV Store has replication issues. See introspection data and mongod.log for details. Local instance has state Recovering..".

What can I do to fix the issue?
When I checked with the kvstore status command for this particular SH member, the status is shown as recovering.
On using the resync command, even then the issue still exists.

Can you please let me know what steps should be followed to rectify the issue.
Will there be any impact on the performance of search heads?

Labels (1)
0 Karma
1 Solution

kranitha
Explorer

Fixed the issue by following below steps:
1. Login to the SHM and navigate to the bin directory . Take a backup of KVstore using the below command
./splunk backup kvstore

  1. Now Clean the kvstore locally by stopping the service ./splunk stop ./splunk clean kvstore –local ----- Accept the scary message as "Y" ./splunk start

View solution in original post

kranitha
Explorer

Fixed the issue by following below steps:
1. Login to the SHM and navigate to the bin directory . Take a backup of KVstore using the below command
./splunk backup kvstore

  1. Now Clean the kvstore locally by stopping the service ./splunk stop ./splunk clean kvstore –local ----- Accept the scary message as "Y" ./splunk start

View solution in original post

denissotoacc
Explorer

When you said "SHM" is it the Deployer, the Search Head Captain or the same SH with the issue?

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!