One of our indexer cluster crashes and CM is showing many messages like
Missing enough suitable candidates to create replicated copy in order to meet replication policy. Missing={ default:1 }
What does this message mean...
This exact question came up earlier today internally, so I am just going to copy in the engineer's response in the hope it helps:
That message means a bucket is missing to meet RF, but it cant fix it at this time. Possible reasons:
1) not enough indexers. if your RF=3, and you have 2 indexers remaining, obviously it's missing a 3rd indexer (default site missing 1 more indexer)
2) existing indexers are already busy replicating. We only do 5 replications at a time per indexer, so it's possible you'll see this message when replications are active and there are no available slots. (example, RF=3, we started with 4 indexers, lost one, then the 3 indexers will be busy replicating to each other and this message will show up for buckets that are in the queue to be fixed up - but other buckets are being actively fixed)
Does this match your failure situation?
I am facing the same issue. RF=2 and have three Indexers and Time in Fixup shows 17 hours.
How can I get this speed up ? Should it be taking this long ?