Hello,
Currently we have a multisite cluster that has two sites. Using the Rest endpoint, we put 33% of our peers in site 2 in detention. We are currently seeing that our search factor is not met. Is this a known issue with detention in multisite clustering? Should manually rebalancing the primary buckets work?
This is the error I'm receiving :
Missing enough suitable candidates to create a replicated copy in order to meet replication policy. Missing={ site2:1 }
Once a server is put into detention, it is essentially removed from the cluster in terms of bucket replication and rebalancing. So if your search / replication factor is no longer met with these servers in detention, then yes your SF wont be made and you will get these errors.
Manually rebalancing should work assuming you have the requisite number of peers in your site SF/RF. See this doc : http://docs.splunk.com/Documentation/Splunk/6.3.3/Indexer/Rebalancethecluster
Once a server is put into detention, it is essentially removed from the cluster in terms of bucket replication and rebalancing. So if your search / replication factor is no longer met with these servers in detention, then yes your SF wont be made and you will get these errors.
Manually rebalancing should work assuming you have the requisite number of peers in your site SF/RF. See this doc : http://docs.splunk.com/Documentation/Splunk/6.3.3/Indexer/Rebalancethecluster