When I run the installer it fails and rolls back
I see errors about MongoDb lockfile
2019-01-14T18:45:28.291Z W - [initandlisten] Detected unclean shutdown - /DB/kvstore/mongo/mongo\mongod.lock is not empty.
2019-01-14T18:45:28.320Z I STORAGE [initandlisten] **************
old lock file: /DB/kvstore/mongo/mongod.lock. probably means unclean shutdown,
but there are no journal files to recover.
this is likely human error or filesystem corruption.
please make sure that your journal directory is mounted.
found 7 dbs.
see: http://dochub.mongodb.org/core/repair for more information
I ran ./mongod --dbpath /DB/kvstore/mongo it repaired and emptied the mongod.lock file. But then when I run the installer again I get the same error.
When it fails I can manually start the splunkd service and it runs 7.2.1
I also see in the migration.log.2019-01-14.16-32-00
ERROR: [App Key Value Store migration] Service(36) terminated before the service availability check could complete. Exit code 100, waited for 0 seconds.
ERROR while running mongod-fix-voting-priority migration.
This can be permissions issue or you may also have to delete the mongod.lock file under $SPLUNKHOME/var/lib/splunk/kvstore/mongo/ and run the mongod --repair command to as well
PFB link in splunk answers, this might help you:
Let me know if this helps.