upgrade to kvstore v7 impossible on MacOS?!

[schose]

Hi all,

When upgrading from v9.4.1 to a newer version (including 10) on MacOS (arm) i receive the error message:

-> Currently configured KVSTore database path="/Users/andreas/splunk/var/lib/splunk/kvstore"
-> Currently used KVSTore version=4.2.22. Expected version=4.2 or version=7.0
CPU Vendor: GenuineIntel
CPU Family: 6
CPU Model: 44
CPU Brand: \x
AVX Support: No
SSE4.2 Support: Yes
AES-NI Support: Yes

There seems to be an issue with determine AVX correctly thru rosetta?! - Anyway, i tried to upgrade on v9.4.1using

~/splunk/bin/splunk  start-standalone-upgrade kvstore -version 7.0 -dryRun true

and receive the error

In handler 'kvstoreupgrade': Missing Mongod Binaries :: /Users/andreas/splunk/bin/mongod-7.0; /Users/andreas/splunk/bin/mongod-6.0; /Users/andreas/splunk/bin/mongod-5.0; /Users/andreas/splunk/bin/mongod-4.4; Please make sure they are present under :: /Users/andreas/splunk/bin before proceeding with upgrade.
Upgrade Path = /Users/andreas/splunk/bin/mongod_upgrade not found  Please make sure upgrade tool binary exists under /Users/andreas/splunk/bin

The error that mongod-4.4, mongod-5.0, mongod-6.0 and mongod-7.0 are missing is correct, the files are not there. There are not in delivered splunk .tgz for macos. The linux tarball includes them.. 

any hints?

best regards,

Andreas

[livehybrid]

Hi @schose 

A full Splunk Enterprise installation is not currently supported for MacOS ARM (See https://help.splunk.com/en/splunk-enterprise/get-started/install-and-upgrade/10.0/plan-your-splunk-e...) - Only the UF package is supported, this could be for a number of reasons however MongoDB 5.0+ requires a CPU with AVX support which Silicon macs do not support (with Intel macs aging out).

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[abancel_]

As per the compatibility matrix, the Developer License is still listed supported with x86 binaries, this should allow running a full Splunk Enterprise instance on Apple silicon with Rosetta (of course within the limitations of the Developer License). Otherwise there would be no point in providing MacOS binaries in the first place.
I can also confirm that a fresh install of 10.0 works but the mongod_upgrade and mongod binaries have been missing from the MacOS tarballs since at least 9.3.0 (if we compare with the linux tarballs).
I'll reach out to Support to get their feedback.

[livehybrid]

Thanks @abancel_ 

So does this mean that running the X86/Intel binaries for MacOS on an ARM Mac is supported? My understanding was that the support was tied to the architecture that the binaries were built for, rather than any emulated architecture. 

I've previously had issues with mongo on both ARM Mac using X86 binaries for MacOS and also using X86 Binaries in a linux docker container using Rosetta. At the time I was advised that these scenarios were not supported.

Thanks  

[abancel_]

Hi @livehybrid ,

That was a sound advice.
I wouldn't go as far as calling this a supported configuration (especially not for a production environment), but running the darwin x86 binaries on Apple silicon is achievable thanks to the Rosetta translation layer, and quite frankly, there is not much overhead noticeable for a single user test/dev environment (especially on a host that may or may not meet the minimum Splunk Enterprise hardware requirements in the first place).

Back to the mongod issue:

mongod-7.0 seems to start just fine with a fresh install of 10.0 (serverVersion : 7.0.18).
But indeed, the mongod_upgrade binary is missing from the darwin_x86 tarball - this means also that the issue would likely be the same on Intel Macs.
I've reached out to our support team and will update if more pointers come out. There should ideally be a way to avoid the vanilla installation.

[isoutamo]

I have reported this to the PM several months ago and he/she promised to look it, but doesn't promise any quick fix 😞

[schose]

Hi @livehybrid,

Well, strange thing is that mongo7 is working for splunk v10 when installing vanilla.

I would strongly guess that the update even does not work on intel macs also, because the mongo binaries needed for migration are not the in macos intel tarball. 

Cheers,

Andreas