Knowledge Management

Why am I getting KV store error 'External command based lookup 'MY_KV_LOOKUP' is not available because KV Store initialization has failed'?

Communicator

Hello,

After a while, my KV Store isnt working.

I receive this message:

The lookup table 'External command based lookup 'MY_KV_LOOKUP' is not available because KV Store initialization has failed. Please contact your system administrator.' is invalid.

Nothings shows wrong when I restart Splunk or in _internal logs.

Tags (1)
1 Solution

Communicator

After manually restarting mongod service, it worked.

View solution in original post

Esteemed Legend

In my case it was that the certificates and other files were too permissive. I did this to fix it:

cd /opt/splunk/var/lib/splunk/kvstore/mongo
chmod 600 *
/opt/splunk/bin/splunk restart
0 Karma

Esteemed Legend

In my case it was that the certificates and other files were too permissive. I did this to fix it:

cd /opt/splunk/var/lib/splunk/kvstore/mongo
chmod 600 *
/opt/splunk/bin/splunk restart
0 Karma

Communicator

After manually restarting mongod service, it worked.

View solution in original post

Splunk Employee
Splunk Employee

Hi, I had the exact same behaviour but in my case the root cause was a firewall between the search head cluster peers that did not allow communication on kvstore replication port (default 8191)

0 Karma

Path Finder

How did you perform your mongod service restart? Thanks.

0 Karma

Communicator

For me the solution was (and still is) to deleted the mongod.lock file and do a "Splunk restart"

0 Karma

Path Finder

I have tried this option also but th issue has not resolved. Is there any alternative

0 Karma

Communicator

Please, How did you restarted mongod service? Are you running splunk on windows or Linux

Thanks

Path Finder

I tried to restart mongod on linux, but it requires some ssl lib file under splunk/lib.

Anyone has tried and successfully restarted mongod service?

0 Karma

Explorer

I am experiencing the same issue. Any lucks on the command for Linux?

0 Karma

New Member

I just experienced the same issue - Splunk 6.2.3 running on Linux. As well as deleting mongod.lock, I also had to delete splunk.key, both under $SPLUNK_HOME/var/lib/splunk/kvstore/mongo, before doing a splunk restart.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!