Splunk Enterprise

KV Store status failed after upgrade to 8.0.4.1?

krylov
Explorer

Good afternoon!

 After upgrade to version 8.0.4.1 from 8.0.1 KV Store status failed.

 

Failed to start KV Store process. See mongod.log and splunkd.log for details.
15.06.2020, 21:12:09
KV Store changed status to failed. KVStore process terminated..
15.06.2020, 21:12:08
KV Store process terminated abnormally (exit code 14, status exited with code 14). See mongod.log and splunkd.log for details.

 

./splunk show kvstore-status

This member:

                                 backupRestoreStatus : Ready

                                      disabled : 0

                                          guid : A5CBF13E-6100-40DA-8C51-541893E5A0A0

                                          port : 8191

                                    standalone : 1

                                        status : failed

 

/data/splunk/var/log/splunk/mongod.log

2020-06-16T05:25:17.315Z W CONTROL  [main] net.ssl.sslCipherConfig is deprecated. It will be removed in a future release.

 2020-06-16T05:25:17.323Z F NETWORK  [main] The provided SSL certificate is expired or not yet valid.

 2020-06-16T05:25:17.323Z F -        [main] Fatal Assertion 28652 at src/mongo/util/net/ssl_manager.cpp 1214

 2020-06-16T05:25:17.323Z F -        [main]

 ***aborting after fassert() failure

 

 

/data/splunk/var/log/splunk/splunkd.log

ERROR KVStoreAdminHandler - An error occurred.

ERROR KVStorageProvider - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling ismaster on '127.0.0.1:8191']

ERROR KVStoreIntrospection - failed to get introspection data

 

 

Could you help me figure out how to fix this?

Thanks in advance!

 

 

Labels (1)
1 Solution

krylov
Explorer

The question isn't longer relevant. The problem solved by the procedure described in this ask

https://community.splunk.com/t5/Knowledge-Management/After-upgrading-to-6-5-0-KV-Store-will-not-star...

View solution in original post

0 Karma

pavankumarh
Path Finder

https://community.splunk.com/t5/Knowledge-Management/Why-is-KV-Store-certificate-renewal-not-working... 

On Windows, you may get the following error message in mongod.log:

Fatal Assertion 50755 at src\mongo\util\net\ssl_manager_windows.cpp 1609

To fix the error that causes mongod to terminate, you need the following in addition to deleting server.pem:

Open Windows certificate management MMC for the local computer ( certlm.msc )
Navigate to Personal > Certificates
Delete any entries named SplunkServerDefaultCert

Restart splunk. 

0 Karma

spodda01da
Path Finder

@krylov : Did you manage to find the solution... I have similar issue where server.pem certificate is valid and also regenerated mongo DB.

0 Karma

krylov
Explorer

The question isn't longer relevant. The problem solved by the procedure described in this ask

https://community.splunk.com/t5/Knowledge-Management/After-upgrading-to-6-5-0-KV-Store-will-not-star...

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...