topic Re: Kvstore failed in Splunk Enterprise Security

Kvstore failed

egko — Wed, 03 Dec 2025 06:49:50 GMT

On my current machine, Kvstore is failing.
When I restart Splunk, the Kvstore status is "Ready." However, when I click the Audit Log tab in ES, the status changes to "Failed." This makes it impossible to access other Kvstore-related functions, such as incident review in ES.

I tried changing the server.pem file to a different extension and restarting, as well as changing the mongod.lock and splunk.key files to different extensions and restarting. I also tried changing all configuration files, but nothing worked.

I'm wondering if there are any other solutions.

Please help.

Splunk version : 8.1.10.1
Splunk Enterprise Security: 7.0.1

ERROR-Log
Failed to execute KVstore lookups External command based lookup 'goverence_lookup' is not available because KVstore initialization has failed, Contact your system admin...
Failed to create kvstore lookup

Re: Kvstore failed

thahir — Wed, 03 Dec 2025 07:26:20 GMT

@egko Take the backup of KV store
ref: https://help.splunk.com/en/data-management/splunk-enterprise-admin-manual/9.4/administer-the-app-key-value-store/back-up-and-restore-kv-store

and try to clean the kvstore and restart the splunk.

check the status

splunk show kvstore-status --verbose

splunk stop
splunk clean kvstore --local
splunk start

please refer the below document as well for more details about kv store troubleshooting

https://help.splunk.com/en/splunk-enterprise/administer/admin-manual/9.0/welcome-to-splunk-enterprise-administration/administer-the-app-key-value-store/kv-store-troubleshooting-tools

Re: Kvstore failed

egko — Wed, 03 Dec 2025 08:19:14 GMT

The mongod.lock file is 0 bytes no matter how many times I restart it.

Re: Kvstore failed

thahir — Wed, 03 Dec 2025 08:21:09 GMT

@egko remove the lock file and clean up the kvstore and do the restart

Re: Kvstore failed

PrewinThomas — Wed, 03 Dec 2025 09:43:33 GMT

@egko

Your Splunk Enterprise 8.1x is outdated, and Splunk Enterprise Security 7.0.1 is likely incompatible, causing the KV Store to fail (from "Ready" to "Failed") when ES loads.

Upgrade Splunk Enterprise, then upgrade ES to a compatible version.

#https://help.splunk.com/en/splunk-enterprise/get-started/install-and-upgrade/9.0/upgrade-or-migrate-splunk-enterprise/how-to-upgrade-splunk-enterprise

Regards,
Prewin
🌟If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!