topic Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"? in All Apps and Add-ons

Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Thu, 14 Jan 2016 10:52:26 GMT

Hello All

When I open the Splunk App for Windows Infrastructure, I get a Prerequisites message: "Key value store must be enabled"

I already deleted the mongo lock file and grant 400 permissions to apps/splunk/var/lib/splunk/kvstore/mongo/splunk.key
and genenrated ssl with 1024

Any ideas what to check next?
Tnx in advance!

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

alemarzu — Thu, 14 Jan 2016 12:58:29 GMT

Hi there mate,

Try this from a shell and then restart Splunk.
chmod 700 /opt/splunk/var/lib/splunk/kvstore/mongo/splunk.key

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Thu, 14 Jan 2016 13:19:50 GMT

Hello
tnx for reply
I already configured 700 permissions then reverted back to 400 with no luck.

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

alemarzu — Thu, 14 Jan 2016 13:38:20 GMT

Take a look at RichING's comments.

https://answers.splunk.com/answers/268584/splunk-app-for-windows-infrastructure-it-has-error.html

And 2nd EDIT from juriggs

https://answers.splunk.com/answers/206030/splunk-app-for-windows-infrastructure-why-am-i-get-1.html

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Thu, 14 Jan 2016 16:17:32 GMT

its not permissions issue i think , maybe mongo db corrupted?

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

alemarzu — Thu, 14 Jan 2016 18:37:21 GMT

I can't tell mate, you may need help from support team.

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

s2_splunk — Thu, 14 Jan 2016 21:38:55 GMT

What error messages are logged in splunkd.log when you open the app? Did you verify that server.conf has the kvstore enabled, i.e. you see disabled=false under the [kvstore] stanza?

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Sat, 16 Jan 2016 12:08:57 GMT

Hi,
I see the kvstore is enabled (disabled=false)

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Sun, 17 Jan 2016 11:27:57 GMT

Hi on other lab server it is enabled , i changed it in
/opt/splunk/etc/system/default/server.conf , kvstore was disabled in one section to true
[introspection:generator:kvstore]
disabled = false (was true)
but on other with same settiings i am get error that is not enabled.
Maybe mongo db is corrupted and needs to be repaired?
Tnx

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

jkat54 — Tue, 29 Sep 2020 08:23:51 GMT

Sounds like your firewall ports are not open. Run netstat -an and look for something like "LISTENING" on 127.0.0.1:(YOUR_KVSTORE_PORT).

If you're "listening" on that port, great, kvstore is enabled and splunk bound to the port successfully upon startup. Now you must test connectivity to that server on that port. An easy way to do this is with telnet.

Another method is using the nc command in linux (netcat).

With splunk shut down, on the server with issues do the following:
1. stop splunk
2. Tell netcat to listen on your kvstore port (binds to the port) nc -l {YOUR_KVSTORE_PORT}
3. From another server with network connectivity to the server in question: nc splunkserver {YOUR_KVSTORE_PORT} example: `nc splunksearchheadip 8191'
4. Both servers will have a blank screen with blinking cursor, now if you type on the terminal on the remote server, you should see words printed on the "problematic" server. This will prove that network connectivity on your kvstore port exists. If this test fails, you will not see anything printed on the screen of the problematic server. That would meant the port isnt open.

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Sun, 17 Jan 2016 13:51:34 GMT

Hi,
Tnx for the reply
with splunk running service i see that port 8191 is listening and telnet works

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Sun, 17 Jan 2016 14:32:17 GMT

Hi
I see this error in mongo.log

Did not find local replica set configuration document at startup; NoMatchingDocument Did not find replica set configuration document in local.system.replset

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

jkat54 — Sun, 17 Jan 2016 18:37:28 GMT

When you say telnet works, you mean you get a blank screen with a blinking cursor?

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

vad34 — Mon, 18 Jan 2016 09:06:18 GMT

right! i tested it from own server

Re: Splunk App for Windows Infrastructure: Why a I getting prerequisites message "Key value store must be enabled"?

jkat54 — Thu, 21 Jan 2016 13:32:24 GMT

Do you have a search head cluster or search head pool? If so, can they telnet to the others just fine on the same kv store port? If it's more than one search head, you shouldnt be testing telnet from the local machine to the local machine, but making it traverse the network path between two machines instead.