Knowledge Management

Why am I getting KV store error 'External command based lookup 'MY_KV_LOOKUP' is not available because KV Store initialization has failed'?

pedromvieira
Communicator

Hello,

After a while, my KV Store isnt working.

I receive this message:

The lookup table 'External command based lookup 'MY_KV_LOOKUP' is not available because KV Store initialization has failed. Please contact your system administrator.' is invalid.

Nothings shows wrong when I restart Splunk or in _internal logs.

Tags (1)
1 Solution

pedromvieira
Communicator

After manually restarting mongod service, it worked.

View solution in original post

woodcock
Esteemed Legend

In my case it was that the certificates and other files were too permissive. I did this to fix it:

cd /opt/splunk/var/lib/splunk/kvstore/mongo
chmod 600 *
/opt/splunk/bin/splunk restart

giorgio79dis
Engager

Thank you! Your solution worked

0 Karma

woodcock
Esteemed Legend

In my case it was that the certificates and other files were too permissive. I did this to fix it:

cd /opt/splunk/var/lib/splunk/kvstore/mongo
chmod 600 *
/opt/splunk/bin/splunk restart
0 Karma

pedromvieira
Communicator

After manually restarting mongod service, it worked.

View solution in original post

jkellerman_splu
Splunk Employee
Splunk Employee

Hi, I had the exact same behaviour but in my case the root cause was a firewall between the search head cluster peers that did not allow communication on kvstore replication port (default 8191)

0 Karma

ashnet16
Path Finder

How did you perform your mongod service restart? Thanks.

0 Karma

bravon
Communicator

For me the solution was (and still is) to deleted the mongod.lock file and do a "Splunk restart"

0 Karma

pasokkum
Path Finder

I have tried this option also but th issue has not resolved. Is there any alternative

0 Karma

cafissimo
Communicator

Please, How did you restarted mongod service? Are you running splunk on windows or Linux

Thanks

kundeng
Path Finder

I tried to restart mongod on linux, but it requires some ssl lib file under splunk/lib.

Anyone has tried and successfully restarted mongod service?

0 Karma

cedarcrestone
Explorer

I am experiencing the same issue. Any lucks on the command for Linux?

0 Karma

peteharverson
New Member

I just experienced the same issue - Splunk 6.2.3 running on Linux. As well as deleting mongod.lock, I also had to delete splunk.key, both under $SPLUNK_HOME/var/lib/splunk/kvstore/mongo, before doing a splunk restart.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!