we have a 3 node SH-Cluster where one member is not getting up again.
If we want to restart the Splunk daemon it will stuck on the very last task to start the web server.
After a while we are getting a WARNING: web interface does not seem to be available!
On the newly selected captain node I've checked the kv status for the specific host:
You can run splunk clean raft on the affected member only, too. See if that helps.
Your two other members are working fine? What are the outputs of splunk show shcluster-status and splunk show kvstore-status on the working members/captain?
In case that only one member is going crazy, I'd suggest simply removing it from the cluster and adding it again after cleaning it if splunk clean raft didn't do the job.