Knowledge Management

SH's KVStore keeping "Initial Sync" status

Splunk Employee
Splunk Employee

Using Splunk v7.1.4 and find that one of SH is keeping "Initial Sync" in replication status of KVStore for few days.

Below logs in mongod.log:

"xxxx-xx-xxT06:58:52.058Z I QUERY [conn5] assertion 13436 not master or secondary; cannot currently read from this replSet member ns:s_SplunkRXskOyCRWNcQsfE7@@TYWgMZ_WA_goaWbaVnkIQFzA7WcgP9a3MEXki.system.profile query:{ $query: { ts: { $gte: new Date(1552707830499) }, millis: { $gte: 1000 } }, $orderby: {} }"

Tried below 3 methods but it doesn't help

1.) splunk clean kvstore -local, restart splunk
2.) splunk clean kvstore -cluster, restart splunk
3.) splunk resync kvstore

Tags (1)
0 Karma

Splunk Employee
Splunk Employee

Encounter a known issue on kvstore in v7.1.x. Splunk Search head cluster initial kv sync continues to fail.

Try to apply the following workaround and it helps.

Add the following attribute for all SH members and restart all SH members

allowUnsafeRenamesDuringInitialSync = true

NOTE: allowUnsafeRenamesDuringInitialSync attribute is only available in mongodb 3.4.10 and is not available prior to Splunk version 7.1.4.

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...