Knowledge Management

Why is KV Store initialization failing on one of our add-on to receive logs?

khusain_splunk
Splunk Employee
Splunk Employee

While setting up one of our add-on to receive logs, we encountered an issue. While reviewing the internal log we found an error (HTTPError: HTTP 503 error Service Unavailable -- KV store initialization failed . This error also shows up every time splunk services are restarted.

0 Karma
1 Solution

khusain_splunk
Splunk Employee
Splunk Employee

Hi,

Please check mongod.log under $SPLUNK_HOME/var/log/splunk/, if it says related to SSL certificate, exp:

The provided SSL certificate is expired or not yet valid.
No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile

Then, you need to renew the SSL certificate. If you are using third-party certificate then place the new certificate and restart splunkd. Else, if you are on default certificate, go under $SPLUNK_HOME/etc/auth/ and rename server.pem file and restart the splunk which will generate the new SSL certificate and kv store will be up .

Thanks
Kashif Husain

View solution in original post

khusain_splunk
Splunk Employee
Splunk Employee

Hi,

Please check mongod.log under $SPLUNK_HOME/var/log/splunk/, if it says related to SSL certificate, exp:

The provided SSL certificate is expired or not yet valid.
No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile

Then, you need to renew the SSL certificate. If you are using third-party certificate then place the new certificate and restart splunkd. Else, if you are on default certificate, go under $SPLUNK_HOME/etc/auth/ and rename server.pem file and restart the splunk which will generate the new SSL certificate and kv store will be up .

Thanks
Kashif Husain

View solution in original post

Mesa_Splunkr
Loves-to-Learn

I am having issues setting up a proofpoint TAP app, here is what the log says.

 -0500 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/TA-Proofpoint-TAP/bin/proofpoint_tap_siem.py" proofpoint_tap_siem://TAP API: stream_events/HTTP 503 Service Unavailable -- KV Store initialization failed. Please contact your system administrator.

I found this article very helpful; however, my certificate is valid, and does not expire till 7/23/2023. My mongod.log also has the following in it.

W CONTROL  No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile parameter

I am checking the date via GUI when I login to the splunk server. I will research more, wanted to post this to see if you can help. Thanks in advance.

 

 

0 Karma

kcooper
Path Finder

I just replaced my certificate and the data from our Azure accounts started ingesting again but then it stopped again. 

Received same error:  HTTP 503 Service Unavailable -- KV Store initialization failed.

Any idea how to fix this issue if the certificate is still active? 

0 Karma

_smp_
Builder

This just saved my a$$. Thanks!

0 Karma