Deployment Architecture

Clustering and reassigning primaries issue

Path Finder

We have 2 peers that each forwarder load balances between so there is roughly 50% of the primary data on each one. The load on the peers is thus evenly spread. The replciation factor of 2 so each peer works as a back up for the other.
Doing a search I can see that split on the "splunk server" field is roughly 50 - 50 which is exactly what I want to see.
During routine maintenance one of the peers is brought offline (splunk offline command) . While it is down the primaries are reassigned to the other peer as you would expect. Looking at "splunk server" from a search I can see 100% of the data on the remaining peer so the resilancy works. The problem is that when the peer is brought back online there is still 100% of the primary data on peer that was left up. This means that one peer is getting hammered while the other does nothing. Is there any way to force splunk to reassign 50% of the primaries back to the peer that was down? I am aware that the data is on the peer but it is now a replicate and does not participate in the searches.

Path Finder

Steve,
Thanks for your help. Any reason to upgrade to 6!

0 Karma

Splunk Employee
Splunk Employee

Have you tried manually rebalancing the cluster, as described here?

http://docs.splunk.com/Documentation/Splunk/6.0/Indexer/Rebalancethecluster

Splunk Employee
Splunk Employee

Tony - I don't believe there's a way to do this in 5.x.

0 Karma

Path Finder

Thanks Steve,

That looks like it a new feature on 6.0 and is exactly what we need. Howver we are on 5.0.5, do you know of a workaround for 5.0.5?

Cheers

Tony

0 Karma

Path Finder

Splunk 5.0.5 by the way.

0 Karma