The problem: My search head is populating with an audit lookup error after upgrading from 9.0.0 to 9.0.2.  What I've found: Looking into windows cert mmc on my Splunk server I saw two certs. The self-signed root CA from Splunk, and a cert named SplunkServerDefaultCert below it that is expired. I'm assuming this expired cert is causing the issue and not the actual upgrade itself. Next, I checked my KVStore status, it's reading "failed."  Then I checked web.conf, enableSplunkWebSSL = true, there's a password populated in sslPassword, then I ensured privateKeyPath/serverCert/sslRootCAPath had the files in each location as well as checked the expiration dates for each one. The PEM for serverCert is indeed expired.  What I've done so far: I renamed the server.pem file to server.pem.back, restarted Splunk and hoped a new cert generated. Didn't work. All that did was prevent the web interface from working.  Then I went into openssl.conf and inserted "extendedKeyUsage = serverAuth, clientAuth" in the [v3_req] settings and uncommented "req_extensions = v3_req"  in [req].  I moved on to openssl to generate a new server cert. Created and signed the new server CSR, verified it, and replaced the  old  server cert w/ the new server PEM. Still didn't work.  Found $SPLUNK_HOME/var/lib/splunk/kvstore/mongo/splunk.key, renamed it, restarted splunk, found that a new key was generated, and my KVstore status still reads as "failed."  Going forward: Not sure what else I can do to fix this. Given I backed up everything, I restored it all back to square one w/all the OG certs and keys except the openssl.cnf, I left the changes I made stated earlier.  This is my first time working w/certs, I'm not too savvy w/ any of it, but a lot of the things I did above have all come from other asked questions on this community.  I think one place I may have made a mistake was signing the server.csr I created. I signed it with the new private.key that was created along with it, not the key that is currently annotated in web.conf. I don't know if that makes a difference, but I can't think of any other reason why the new server.pem  didn't work.  For reference: Jeremy describes my exact issue in the below post; however, I do not have the password to the OG splunk cert in the mmc, so I cannot recreate it as he did.  Windows upgrade from 8.1.1 to 9.0: Why does it fai... - Splunk Community Additionally, the above case, is the exact issue I am having down to the error codes. ... View more