KVstore error output after splunk enterprise upgrade from 9.1.1 to 9.4.1

[swamybhatta]

./splunk cmd mongod -version

mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod:/splunk/lib/libssl.so.10: no version information available (required by mongod)
db version v7.0.14
Build Info: {
"version": "7.0.14",
"gitVersion": "ce59cfc6a3c5e5c067dca0d30697edd68d4f5188",
"openSSLVersion": "OpenSSL 1.0.2zk-fips 3 Sep 2024",
"modules": [
"enterprise"
],
"allocator": "tcmalloc",
"environment": {
"distmod": "rhel70",
"distarch": "x86_64",
"target_arch": "x86_64"
}
}

why am i getting  mongo db error 

mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod:/splunk/lib/libssl.so.10: no version information available (required by mongod)

[sushilkar]

mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod:/splunk/lib/libssl.so.10: no version information available (required by mongod)

isn't a harmful error that you should worry about.

Splunk ships its own libcrypto.so.10 that doesn’t have the GNU symbol version sections.The dynamic loader warns you, but execution usually continues. 

You could check if the mongod is seeing that lib from /opt/splunk/lib overridden path:

Simply that error you can ignore as mentioned above.

If loading issue is there: you will see the lib missing

$ SPLUNK_HOME=/opt/splunk
$ LD_LIBRARY_PATH=$SPLUNK_HOME/lib:$LD_LIBRARY_PATH ldd $SPLUNK_HOME/bin/mongod 

moreover

$ SPLUNK_HOME=/opt/splunk
$ LD_LIBRARY_PATH=$SPLUNK_HOME/lib:$LD_LIBRARY_PATH $SPLUNK_HOME/bin/mongod --version 

OR

splunk cmd mongod -version this works for you means no issues with KVstore 7.0

Do you see the process mongod not started by SH?

pgrep -f mongod , can see if its running 

also check the kvstore-status 

/opt/splunk/bin/splunk show kvstore-status

[GSNRMUVW]

In the Splunk UI appears an error...

KV Store process terminated abnormally (exit code 4, status PID 55313 killed by signal 4: Illegal instruction). See mongod.log and splunkd.log for details.

KV Store changed status to failed. KVStore process terminated..

Failed to start KV Store process. See mongod.log and splunkd.log for details.

[tej57]

Hey @GSNRMUVW,

Did you check the mongod.log? It would indicate some message as an error for terminating the KVStore process. Can you paste the last few lines from the log and mask the sensitive data?

Thanks,
Tejas. 

[GSNRMUVW]

splunk@debian:~$ tail -n50 /opt/splunk/var/log/splunk/mongod.log
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libssl.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libssl.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libssl.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libssl.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libssl.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libcrypto.so.10: no version information available (required by mongod)
mongod: /opt/splunk/lib/libssl.so.10: no version information available (required by mongod)

In my case, the problem exists since some kernel updates like debian 12.11 and debian 13.0...

https://lists.debian.org/debian-testing/2025/07/msg00035.html

link-systemctl-shared=true/false

[sushilkar]

The logs that you see are not harmful, treat it like no version info added library .so file in /opt/lib/splunk its more of a warning. Check if you are able to run mongod --version. (https://community.splunk.com/t5/forums/replypage/board-id/core-splunk-enterprise/message-id/22866)

Check mongod --version you are able to run, see this post. 

https://community.splunk.com/t5/Splunk-Enterprise/Issue-with-kvstore-in-version-9-4-3/m-p/751436#M22...

Sa

[GSNRMUVW]

root@debian:~# /opt/splunk/bin/mongod --version
/opt/splunk/bin/mongod: error while loading shared libraries: libsasl2.so.3: cannot open shared object file: No such file or directory

[livehybrid]

Hi @swamybhatta 

The mongo command isnt intended to be run directly from the CLI - The error you are seeing is likely due to the missing ENV variables such as  LD_LIBRARY_PATH.

Does mongo give an error when the processes is created by Splunk as part of the service's boot sequence?

🌟 Did this answer help you? If so, please consider:

• Adding kudos to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing