After a RAID failure, the disks were supposedly not impacted and a fsck has been completed, but since then, I am seeing the below in splunkd_stderr.log each time I try to start. Seems to be fatal as there are no splunk threads running afterwards.
2015-04-07 16:44:41.288 -0500 splunkd started (build 245427)
terminate called after throwing an instance of 'std::runtime_error'
what(): could not parse raft entry file
Had the same pb after a filesystem full situation on the filesystem where splunk/var was.
After freeing space, splunk would crash at start
in $SPLUNK_HOME/var/lib/splunk, latest file are a crash file and splunk_stderr
splunk_stderr contains "could not parse raft entry file"
mv $SPLUNK_HOME/var/run/splunk/_raft $SPLUNK_HOME/var/run/splunk/_raft_KO followed by restart fixed it
you need to restart another time again to have no warning by the init script
splunk shcluster-status is also all good again.
You can remove the whole $splunk_home$/var/run/splunk/raft/* structure on that instance and restart it. As long and it's SHC configuration is still valid, when it restarts it will join the SHC. It should automatically rebalance out, if not, a rolling-restart should fix it.