What happens if during the Multisite Search Head Cluster split brain situation, we statically assign a captain on the secondary site, too?

[ikulcsar]

Hi,

I have a theoretical question about Search Head Cluster(SHC) operation in a multisite environment:
- We have multiple sites.
- We have a multisite indexer cluster.
- We have an SHC cluster with nodes on both sites with Enterprise Security for example.

Situation: the two site lost the connection between them.
- The primary site can select a new captain. Life goes on on this site:)
- A captain cannot be selected dynamically on the secondary site. Just ad-hoc searches can be run.

The question: what happens if during the split brain situation we statically assign a captain on the secondary site, too (or VMware boots up all the server on both sides)? There will be 2 captains. Data accelerations, ES correlations will run and fill up indexes, summary tables, etc on the local indexer cluster on both sites.
After a recovery, can IXC merge this tables, indexer or it will throw some interesting errors? Will it accelerate, summary events twice and messed up all the notable events?

I know, the double captain is not supported, but in some environment, it can happen...

Regards,
Istvan

[deepashri_123]

hey ikulcsar,
At no point will there be captains set twice,captain is elected internally between the search head cluster, even if all the servers are boot up, the captain will be elected between the cluster members , so there will be no double captaincy.

[ikulcsar]

Hey,

Thank you for your fast response.

I can understand in normal operations condition, there cannot be two captains. But, when the two site separated totally because a network failer, AND on the smaller site (with less than half of the SH nodes), a captain can be delegated manually. I didn't that it is a rational action, but it can be done.

regards
Istvan