Knowledge Management

Splunk is not coming up due to MongoDB exit code 100

keio_splunk
Splunk Employee
Splunk Employee

Splunk is not starting up when performing an upgrade to Splunk 7.1.6 on the indexer.

Error message when starting up Splunk:
mongod exited abnormally(exit code 100, status: exited with code 100) - look at mogod.log to investigate. 
ERROR: Failed to fix non-voting members Priority value during mogod version migration from 3.0.x to 3.4.x.
Reason: ERROR while running mogod-fix-voting-priority migration. 

alt text

0 Karma
1 Solution

keio_splunk
Splunk Employee
Splunk Employee

KVStore and MongoDB was down before the upgrade was initiated.

Repair the KV Store to bring up MongoDB and restart Splunk:
1. Backup $SPLUNK_DB/kvstore/mongo/.
2. Execute: $SPLUNK_HOME/bin/splunk clean kvstore --local
3. Manually delete folder: $SPLUNK_DB/kvstore/mongo/journal 
4. Manually remove $SPLUNK_DB/kvstore/mongo/mongod.lock 
5. Run: $SPLUNK_HOME/bin/mongod --dbpath $SPLUNK_DB/kvstore/mongo --port=8191 --repair 
6. Restart Splunk after the KV Store is repaired.

View solution in original post

keio_splunk
Splunk Employee
Splunk Employee

KVStore and MongoDB was down before the upgrade was initiated.

Repair the KV Store to bring up MongoDB and restart Splunk:
1. Backup $SPLUNK_DB/kvstore/mongo/.
2. Execute: $SPLUNK_HOME/bin/splunk clean kvstore --local
3. Manually delete folder: $SPLUNK_DB/kvstore/mongo/journal 
4. Manually remove $SPLUNK_DB/kvstore/mongo/mongod.lock 
5. Run: $SPLUNK_HOME/bin/mongod --dbpath $SPLUNK_DB/kvstore/mongo --port=8191 --repair 
6. Restart Splunk after the KV Store is repaired.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...