Splunk Enterprise

Storage Engine did not migrate to WiredTiger after upgrade to Splunk 9.0.2?

SplunkNinja
Explorer

Hello everybody!

My standalone SH Storage Engine did not migrate to WiredTiger after upgrade to Splunk 9.0.2.  I then followed these steps: https://docs.splunk.com/Documentation/Splunk/8.1.3/Admin/MigrateKVstore

Migrate the KV store after an upgrade to Splunk Enterprise 8.1.* or 8.2.* in a single-instance deployment

Stop Splunk Enterprise. Do not use the -f option.
Open server.conf in the $SPLUNK_HOME/etc/system/local/ directory.
Edit the storageEngineMigration setting to match the following example:

[kvstore]
storageEngineMigration=true

Save the server.conf file.
To begin the migration, use the following command:

splunk migrate kvstore-storage-engine --target-engine wiredTiger

Starting KV Store storage engine upgrade:
Phase 1 (dump) of 2:
..ERROR: Failed to migrate to storage engine wiredTiger, reason=

How can I troubleshoot this further?  Thanks.

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

TBH, I'm more concerned about the "connection rejected" errors than the "connecting with a certificate" warning, but either way I don't have a solution for you. I suggest double-checking all of your KVstore settings. Contact Splunk Support if you have an entitlement.

---
If this reply helps you, Karma would be appreciated.

SplunkNinja
Explorer

Thanks Rich.

I have opened a Support case to further investigate.

0 Karma

SplunkNinja
Explorer

Hello Rich,

I followed the 9.0.2 steps with the same results.  My mongodb.log is showing issues with SSL.  Not sure why we see - connecting with a certificate with cluster membership

This is supposed to be a standalone SH.  Not sure what to try next.  Thanks.

2022-11-29T19:21:34.576Z W ACCESS [conn1] Client isn't a mongod or mongos, but is connecting with a certificate with cluster membership
2022-11-29T19:21:34.576Z I ACCESS [conn1] Successfully authenticated as principal CN=a.b.c.d,OU=xxx,O=yyy on $external from client 127.0.0.1:37278
2022-11-29T19:21:34.576Z I NETWORK [conn1] end connection 127.0.0.1:37278 (0 connections now open)
mongodump 2022-11-29T19:21:35.588+0000 WARNING: --sslAllowInvalidCertificates and --sslAllowInvalidHostnames are deprecated, please use --tlsInsecure instead
2022-11-29T19:21:35.593Z I NETWORK [listener] connection accepted from 127.0.0.1:37284 #2 (1 connection now open)
2022-11-29T19:21:35.593Z I NETWORK [listener] connection accepted from 127.0.0.1:37286 #3 (2 connections now open)
2022-11-29T19:21:35.605Z E NETWORK [conn2] no SSL certificate provided by peer; connection rejected
2022-11-29T19:21:35.606Z I NETWORK [conn2] Error receiving request from client: SSLHandshakeFailed: no SSL certificate provided by peer; connection rejected. Ending connection from 127.0.0.1:37284 (connection id: 2)
2022-11-29T19:21:35.606Z I NETWORK [conn2] end connection 127.0.0.1:37284 (1 connection now open)
2022-11-29T19:21:35.606Z E NETWORK [conn3] no SSL certificate provided by peer; connection rejected
2022-11-29T19:21:35.606Z I NETWORK [conn3] Error receiving request from client: SSLHandshakeFailed: no SSL certificate provided by peer; connection rejected. Ending connection from 127.0.0.1:37286 (connection id: 3)

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try again using the documentation for the version you are running.  https://docs.splunk.com/Documentation/Splunk/9.0.2/Admin/MigrateKVstore

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...