Knowledge Management

Splunk is not coming up due to MongoDB exit code 100

keio_splunk
Splunk Employee
Splunk Employee

Splunk is not starting up when performing an upgrade to Splunk 7.1.6 on the indexer.

Error message when starting up Splunk:
mongod exited abnormally(exit code 100, status: exited with code 100) - look at mogod.log to investigate. 
ERROR: Failed to fix non-voting members Priority value during mogod version migration from 3.0.x to 3.4.x.
Reason: ERROR while running mogod-fix-voting-priority migration. 

alt text

0 Karma
1 Solution

keio_splunk
Splunk Employee
Splunk Employee

KVStore and MongoDB was down before the upgrade was initiated.

Repair the KV Store to bring up MongoDB and restart Splunk:
1. Backup $SPLUNK_DB/kvstore/mongo/.
2. Execute: $SPLUNK_HOME/bin/splunk clean kvstore --local
3. Manually delete folder: $SPLUNK_DB/kvstore/mongo/journal 
4. Manually remove $SPLUNK_DB/kvstore/mongo/mongod.lock 
5. Run: $SPLUNK_HOME/bin/mongod --dbpath $SPLUNK_DB/kvstore/mongo --port=8191 --repair 
6. Restart Splunk after the KV Store is repaired.

View solution in original post

keio_splunk
Splunk Employee
Splunk Employee

KVStore and MongoDB was down before the upgrade was initiated.

Repair the KV Store to bring up MongoDB and restart Splunk:
1. Backup $SPLUNK_DB/kvstore/mongo/.
2. Execute: $SPLUNK_HOME/bin/splunk clean kvstore --local
3. Manually delete folder: $SPLUNK_DB/kvstore/mongo/journal 
4. Manually remove $SPLUNK_DB/kvstore/mongo/mongod.lock 
5. Run: $SPLUNK_HOME/bin/mongod --dbpath $SPLUNK_DB/kvstore/mongo --port=8191 --repair 
6. Restart Splunk after the KV Store is repaired.

Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...