topic Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"? in Deployment Architecture

Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

chaker — Thu, 12 Feb 2015 06:28:56 GMT

Hello all,

I get the following error message when bootstraping my Search Head Cluster Captain:

In handler 'shclustermemberconsensus': Raft not initialized. This means that dynamic captain mode was not set in server.conf

./splunk bootstrap shcluster-captain -servers_list "https://x.x.x.x:8089, https://x.x.x.x:8089, https://x.x.x.x:8089, https://x.x.x.x:808"

I have connectivity between SH Heads and Captain, and have followed the documentation. All the searchheads have been initialized correctly and report:
Search head clustering has been initialized on this node.
You need to restart the Splunk Server (splunkd) for your changes to take effect.

Here is the output from a show cluster status on the deployer

root@Deployer:/opt/splunk/bin# ./splunk show shcluster-status
In handler 'shclusterstatus': Search Head Clustering is not enabled on this node. REST endpoint is not available

Dose anyone know why RAFT is not initialized? It's a fresh 6.2 64bit Linux tar of Splunk Enterprise.

Thanks in advance,
Chaker

Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

esix_splunk — Fri, 13 Feb 2015 01:00:45 GMT

Have you performed the restart on all of the members and has a captain been succesfully nominated?

./splunk show shcluster-status from the members.

Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

chaker — Mon, 28 Sep 2020 18:54:22 GMT

I'm glad this was brought up again. I had been incorrectly trying to bootstrap the Deployer as a captain. As arahut_splunk mentions below, the Deployer is not part of the cluster, and should not be a captain. The correct answer to my original question is provided below by arahut_splunk.

The splunk show shcluster-status should not be run on the deployer, it needs to be run on one of the 3 cluster members.
1.think of the deployer as a separate entity outside the shcluster which needs to be able to communicate with the shcluster (using secret /pass4SymmKey)
2.the init step needs to be run on the 3 members. The init step should leave a [shclustering] stanza in your server.conf with correct settings.
3.Bootstrap then needs to be run on only 1 of the members.
4.As mentioned before the status needs to be checked on only the members.

Old Answer
My mistake was not initializing the Deployer with the ./splunk init shcluster-config command. I was trying to ./splunk bootstrap shcluster-captain without the prior command.
I misinterpreted the documentation when reading it.
http://docs.splunk.com/Documentation/Splunk/6.2.1/DistSearch/SHCdeploymentoverview
"This instance cannot be a member of the search head cluster"
I misinterpreted this to mean not to initialize the deployer as a cluster member.

Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

a212830 — Tue, 10 Mar 2015 18:32:21 GMT

Should this command be run on the deployer? I asked this question recently and was told no, but now I'm having the same error message.

Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

Steve_G_ — Tue, 10 Mar 2015 18:40:38 GMT

No. You should not run the init shcluster-config command on the deployer. That command is for initializing cluster members only. A deployer cannot be a cluster member.

Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

a212830 — Tue, 10 Mar 2015 19:08:13 GMT

OK, so how does one address the " 'shclustermemberconsensus': Raft not initialized. This means that dynamic captain mode was not set in server.conf" error message. I don't see that listed in my training notes anywhere.

Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

arahut_splunk — Tue, 10 Mar 2015 20:54:00 GMT

The splunk show shcluster-status should not be run on the deployer, it needs to be run on one of the 3 cluster members.

think of the deployer as a separate entity outside the shcluster which needs to be able to communicate with the shcluster (using secret /pass4SymmKey)
the init step needs to be run on the 3 members. The init step should leave a [shclustering] stanza in your server.conf with correct settings.
Bootstrap then needs to be run on only 1 of the members.
As mentioned before the status needs to be checked on only the members.

If you still have issues if you follow the steps, please holler.

Re: Why am I getting Search Head Cluster Captain bootstrap error "Raft not initialized"?

vnguyen46 — Thu, 31 Oct 2019 00:45:29 GMT

Before bootsrap, don't forget to restart all your splunk shcluster members after bring up and initialize. It worked for me after restarted.