topic Re: KVStore error in Monitoring Splunk

KVStore error

Germaine1989 — Thu, 13 Mar 2025 09:28:48 GMT

We get these messages. For exmaple dbconnect doesn't work anymore... how could i solve this?

03-11-2025 12:09:07.792 +0100 WARN MongoClient [1244 KVStoreUpgradeStartupThread] - Disabling TLS hostname validation for localhost

03-11-2025 12:09:07.843 +0100 INFO KVStoreConfigurationProvider [1244 KVStoreUpgradeStartupThread] - KVSTore peer=127.0.0.1:8191 replication state=KV store captain. Health state=1

03-11-2025 12:09:07.843 +0100 INFO MongoUpgradePreChecks [1244 KVStoreUpgradeStartupThread] - Supported Upgrade 3

03-11-2025 12:09:11.773 +0100 ERROR PersistentScript [2200 PersistentScriptIo] - From {"C:\Program Files\Splunk\bin\Python3.9.exe" "C:\Program Files\Splunk\Python-3.9\Lib\site-packages\splunk\persistconn\appserver.py"}: File "C:\Program Files\Splunk\Python-3.9\lib\logging\handlers.py", line 115, in rotate

Re: KVStore error

livehybrid — Thu, 13 Mar 2025 18:34:13 GMT

Hi @Germaine1989

Have you recently upgraded or changed anything in your deployment? Please can you confirm the version and OS you’re running?

Thanks

Re: KVStore error

livehybrid — Thu, 13 Mar 2025 18:41:55 GMT

Check KV Store Status

Open Command Prompt as Administrator.
Navigate to your Splunk bin directory:

cd C:\Program Files\Splunk\bin
Run the following command to check KV Store status:

splunk show kvstore-status
If the KV Store is running, you will see a status message indicating it is ready.

Let us know what this shows.

Also - I assume you've already tried restarting Splunk?

I see you've posted errors from splunkd.log - are there any other logs relating to mongo/KV Store, or anything in mongod.log?

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

Re: KVStore error

Germaine1989 — Fri, 14 Mar 2025 09:18:05 GMT

yes we upgraded to 9.3 on windows server 2019

Re: KVStore error

Germaine1989 — Fri, 14 Mar 2025 09:18:50 GMT

it shows this.. but everything is running

Re: KVStore error

livehybrid — Fri, 14 Mar 2025 10:12:15 GMT

Hi @Germaine1989

So you say that Splunk is running on this server but the kvstore-status check says Splunk isnt running??

Is Splunk definitely not installed into 2 locations on that server (e.g. onto another drive/partition/location) ?

If you run restart Splunk and then run that command, what do you get?

cd C:\Program Files\Splunk\bin splunk restart

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

Re: KVStore error

Germaine1989 — Fri, 14 Mar 2025 10:30:55 GMT

i restart it and tried again.. now i get this message

Re: KVStore error

Germaine1989 — Wed, 19 Mar 2025 09:22:52 GMT

yes we upgraded to 9.3 on windows server 2019