How is bucket deletion due to retention managed in a multisite indexer clustering environment?

[rbal_splunk]

I have a multisite cluster. When some of the buckets meet retention, will the bucket get deleted from all the Cluster Peers? How is the Bucket deletion managed due to Retention in a Multi_site Cluster?

[rbal_splunk · Accepted solution]

Once a bucket is deleted from a peer, it notifies the Cluster Master, and as the Cluster Master marks it frozen, it then alerts to all other peers that it's 'frozen'. This secondary alert doesn't do anything to the bucket, but the remaining peers remember it's frozen for the purpose of a restart. When the bucket is re-added, the CM will still know it's frozen, otherwise the frozen flag may be lost.

Frozen buckets are deleted individually by their indexers. A frozen bucket on indexer-A deletes it off indexer-A, but will not delete it from indexer-B. This is by design. Retention policies kick in individually by indexer.

Once a bucket is frozen, it is marked as so and they no longer count towards a cluster's Searchability, Replication Factor, and Search Factor needs.
If the cluster is unsearchable or not meeting RF/SF requirements, it is not due to buckets marked as frozen.