Deployment Architecture

Remove Site from Indexer Cluster

stepheneardley
Explorer

I've just finished adding new physical indexers to our existing multi-site indexer cluster and I'm trying to figure out the safest method for removing the old virtual indexers.

We started off with a 2 site cluster with each site having 3 members and the following config.

available_sites = site1,site2
multisite = true
replication_factor = 2
site_replication_factor = origin:1,total:2
site_search_factor = origin:1,total:2

When I added the new indexers I created 2 new sites (3 and 4) and amended the config as follows.

available_sites = site1,site2,site3,site4
multisite = true
replication_factor = 4
site_replication_factor = origin:1,total:4
site_search_factor = origin:1,total:4

Now that the upgraded cluster has equalized I'm trying to figure out what the safest method for removing sites 1 and 2 is.  I think it should be;

1. splunk offline --enforce-counts (while watching the indexer clustering dashboard on the CM waiting for all data to be searchable before offlining the next).

2. Put the cluster into maintenance mode and update server.conf on the CM as follows;

available_sites = site3,site4
multisite = true
replication_factor = 2
site_replication_factor = origin:1,total:2
site_search_factor = origin:1,total:2

3. Disabled maintenance mode.

Any and all thoughts/past experiences appreciated.

Labels (1)
0 Karma

garyjohnson48
Explorer

Before you take them offline, change your available sites to site3, site4 so that the CM is operating only on those sites.  Run splunk offline for 24hrs on site1 and site2 before --enforce-counts to make sure everything is running smooth with your new setup. 

Your single-site replication factor shouldn't drop below the lowest amount of indexers you have on a particular site so instead of dropping the replication factor to 2, set it as 3 since that is the number of indexers you have in the site. If you set it as two you will get errors. 

 

Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...