Okay, from my experiences with dealing with this, this basically triggers when the communication between the CM and the IDXs drop off or are not in a sustainable way.
What actually happens is the CM thinks the IDX (Batch Adding one) cannot perform indexing operations hence it starts to move the IDX off the CM.
We all know that when an IDX drops off the CM, its primary buckets gets distributed to the other IDXs on site. By the time this process initiates/completes, the IDX gets back in a healthy communication position with the CM and hence the Batch Adding triggers.
What i did after this scenario was :
Raised the Specs of the CM as we were under spec
Increased the below specs in server.conf
[clustering] [Cluster Master]
service_interval = 10
heartbeat_timeout = 1800
cxn_timeout = 300
send_timeout = 300
rcv_timeout = 300
max_peer_build_load = 5
(using cluster-bundle deployment, no restart is required)
----server.conf-------- [Indexer]
On the Indexers:
cxn_timeout = 600
send_timeout = 600
rcv_timeout = 600
heartbeat_period = 10
Doing this reduced the instance from this occurring and this has stopped completely.
... View more