Splunk Enterprise

KV Store status failed after upgrade to 8.0.4.1?

krylov
Explorer

Good afternoon!

 After upgrade to version 8.0.4.1 from 8.0.1 KV Store status failed.

 

Failed to start KV Store process. See mongod.log and splunkd.log for details.
15.06.2020, 21:12:09
KV Store changed status to failed. KVStore process terminated..
15.06.2020, 21:12:08
KV Store process terminated abnormally (exit code 14, status exited with code 14). See mongod.log and splunkd.log for details.

 

./splunk show kvstore-status

This member:

                                 backupRestoreStatus : Ready

                                      disabled : 0

                                          guid : A5CBF13E-6100-40DA-8C51-541893E5A0A0

                                          port : 8191

                                    standalone : 1

                                        status : failed

 

/data/splunk/var/log/splunk/mongod.log

2020-06-16T05:25:17.315Z W CONTROL  [main] net.ssl.sslCipherConfig is deprecated. It will be removed in a future release.

 2020-06-16T05:25:17.323Z F NETWORK  [main] The provided SSL certificate is expired or not yet valid.

 2020-06-16T05:25:17.323Z F -        [main] Fatal Assertion 28652 at src/mongo/util/net/ssl_manager.cpp 1214

 2020-06-16T05:25:17.323Z F -        [main]

 ***aborting after fassert() failure

 

 

/data/splunk/var/log/splunk/splunkd.log

ERROR KVStoreAdminHandler - An error occurred.

ERROR KVStorageProvider - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling ismaster on '127.0.0.1:8191']

ERROR KVStoreIntrospection - failed to get introspection data

 

 

Could you help me figure out how to fix this?

Thanks in advance!

 

 

Labels (1)
1 Solution

krylov
Explorer

The question isn't longer relevant. The problem solved by the procedure described in this ask

https://community.splunk.com/t5/Knowledge-Management/After-upgrading-to-6-5-0-KV-Store-will-not-star...

View solution in original post

pavankumarh
Path Finder

https://community.splunk.com/t5/Knowledge-Management/Why-is-KV-Store-certificate-renewal-not-working... 

On Windows, you may get the following error message in mongod.log:

Fatal Assertion 50755 at src\mongo\util\net\ssl_manager_windows.cpp 1609

To fix the error that causes mongod to terminate, you need the following in addition to deleting server.pem:

Open Windows certificate management MMC for the local computer ( certlm.msc )
Navigate to Personal > Certificates
Delete any entries named SplunkServerDefaultCert

Restart splunk. 

0 Karma

spodda01da
Path Finder

@krylov : Did you manage to find the solution... I have similar issue where server.pem certificate is valid and also regenerated mongo DB.

0 Karma

krylov
Explorer

The question isn't longer relevant. The problem solved by the procedure described in this ask

https://community.splunk.com/t5/Knowledge-Management/After-upgrading-to-6-5-0-KV-Store-will-not-star...

Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...