Failed to start KV store

[corti77]

Hi,

I run splunk 9.0.8 and after an issue with our storage (LUN full). I had to full scan the disk and successfully repaired the filesystem.

splunk starts now but there is a persistent error with the KV store. the status is the following:

show kvstore-status

This member:
backupRestoreStatus : Ready
disabled : 0
guid : E59FAAA8-D66A-498E-8DDF-0F5C29866F95
port : 8191
standalone : 1
status : failed
storageEngine : wiredTiger

Any suggestion on how could get an operational status?

many thanks.

Jose

[corti77]

just voted, thanks a lot

[gcusello · Accepted solution]

Hi @corti77 ,

you have to reset the mongod.lock:

• stop splunkd
• delete $SPLUNK_HOME\var\lib\splunk\kvstore\mongo\mongod.lock"
• start splunkd

I asked to Splunk ideas to create a script od a dedicated solution to this frequent issue, if you think as me that's important, please vote for this idea at https://ideas.splunk.com/ideas/EID-I-2058

Ciao.

Giuseppe

[Ogorek]

Hello, what to do when my kvstore folder just vanished? It do not create again after restart. Tried to create new folder with splunk privileges but it wont help. Do u have any idea how to reapir this?