Deployment Architecture

Is there a way to stop primary bucket re-assignement when the cluster peers are restarted?

lmcmipl
Explorer

When the cluster peers are restarted, primary buckets are be re-assigned by the cluster master. Is there any way to stop the re-assignment from occurring for a planned peer restart?

Replication Factor = 2 and Search Factor = 2

I am running Splunk 6.2.3

lmcmipl
Explorer

I have replication_factor = 2 and search_factor = 2.

According to the documentation "When you start or restart a master or peer node, the master rebalances the set of primary bucket copies across the peers.... to achieve rebalancing, the master reassigns the primary state from existing bucket copies to searchable copies".

If I set search_factor=1, instead of 2, will rebalancing occur when an indexer is restarted and comes up before the heartbeat timeout ? I won't have searchable copies on the other indexers, so shouldn't that stop the primary re-assignment from occurring?

There really should be a property file that we could use to turn rebalancing off or on.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

What are you trying to achieve by avoiding reassignment?

0 Karma

lmcmipl
Explorer

I have a replication factor of 2, but am also using data model acceleration. Since the feature to replicate accelerated data model tsidx files is not available in Splunk yet, every indexer restart means that the re-assigned primary buckets need to be re-accelerated. This is a huge resource drain and can take days in a large system. It would be better if the primaries were not re-assigned - then I would not have to rebuild the accelerated data model tsidx files.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

I understand this issue well, and empathize. But, data model acceleration replication should be here "soon". For now, I think you're stuck.

lmcmipl
Explorer

Thanks for your response.

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi lmcmipl, I don't expect so. The cluster master does the primary reassignment as an attempt to maintain as close to 100% searchable data as it can. You can make this process more graceful by doing a "splunk offline" on the peer.

Please let me know if this answers your question 😄

Get Updates on the Splunk Community!

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI! Discover how Splunk’s agentic AI ...