@pwrdwn are you running the "splunk migrate kvstore-storage-engine --target-engine wiredTiger"  migration command after installing update 9 or prior to? I initially couldn't get the command  to work prior to the update so I tried after where I received the same error you posted. I reverted snapshot and then got it to run prior to installation of update 9 (was having a cert path issue in my server.conf file causing the initial failure).    ... View more

@nkaru no, I'd reverted to a snapshot prior to the V9 Update and then ran the MSI as the service account. ... View more

@nkaru yes I ran the MSI package for Update 9 as the service account for the SplunkD service ... View more

@Sh4ne0  Finally got this issue cleared. Looks like it was an issue with service account permissions. The steps below are what worked for me.  -I granted the service account used to run Splunkd service full control permissions to the entire Splunk home directory.  -Granted the service account permissions to the Private Keys for the server certificate in the cert store. -When I ran the MSI for Version 9 I ran it as the service account for Splunkd service.  After doing all this the mongod log shows successful authentication using the server certificate and the KVStore starts without error. I'm not 100% sure exactly which one of the steps above corrected the issue but hopefully it works for you as well. I'm still having an issue on one windows Heavy Forwarder where the KVStore failed to upgrade to the latest version but is still initializing fine.  ... View more

I am having almost the exact same issue, the only minor difference is instead of: "InvalidSSLConfiguration: Could not read private key attached to the selected certificate, ensure it exists and check the private key permissions" I have an output of "InvalidSSLConfiguration: Could not find private key attached to the selected certificate, ensure it exists and check the private key permissions" I have verified the certificate and key are in the correct locations, have appropriate permissions and the certificate is not expired. This is on windows server 2019 standard.  ... View more