Deployment Architecture

Why are my sites unbalanced?

w199284
Explorer

I have two sites in a multi-site cluster. 7 peers in each site. I have been running this configuration for two years. Over the last couple months I've begun to notice that one site has been adding more buckets that the other. In approximate numbers, site1 has 203,000 buckets. Site2 has 175,000 buckets. I've run rolling-restart a couple times and data rebalance too. I've searched for troubleshooting tips for situations like these but I have not turned up much useful information. My forwarders (heavy and universal) are configured to auto balance across all peers. I'm on version 7.04 and have been there since shortly after its release. What is going on? Thank you for your consideration.

0 Karma

vishaltaneja070
Motivator

Hello @w199284

I think the issue here is Splunk try to achieve 90% rebalancing not fully 100%. If you want 100% rebalancing on both the sides, then you can achieve that as well using the below command:

splunk edit cluster-config -mode master -rebalance_threshold 1 -auth admin:your_password

The below link can help you better:

https://docs.splunk.com/Documentation/Splunk/7.2.3/Indexer/Rebalancethecluster
0 Karma

ssadanala1
Contributor

How many excess buckets you have ?

0 Karma

w199284
Explorer

No excess buckets.

0 Karma

w199284
Explorer

No excess buckets

0 Karma

adonio
Ultra Champion

please share your replication configurations in server.conf:
site_replication_factor = <comma-separated string>

0 Karma

w199284
Explorer

site_replication_factor = origin:1,total:2

I should mention that it has been configured this way since I went to multi-site.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...