Installation

Can we use the two sites of a multisite indexer cluster to improve the Splunk upgrade?

ddrillic
Ultra Champion

I wonder whether during a Splunk upgrade we can use our two sites of the multisite indexer cluster to improve the Splunk upgrade. Meaning, is it possible to upgrade the indexers of one site and then upgrade the indexers of the second site?

In our upgrade last week we simply upgraded all the indexers together and it took the indexers over 12 hours to catch up with the indexing load, replication and buckets fix-up.

It relates to Large amount of buckets that need to be fixed after the upgrade?

Labels (2)
1 Solution

FrankVl
Ultra Champion

Yes, you could have done a site-by-site upgrade: https://docs.splunk.com/Documentation/Splunk/7.3.0/Indexer/Upgradeacluster#Site-by-site_upgrade_for_...

And if you're already on a recent enough version (7.1.x or newer) you can even do a rolling upgrade of your indexers: https://docs.splunk.com/Documentation/Splunk/7.3.0/Indexer/Searchablerollingupgrade

View solution in original post

FrankVl
Ultra Champion

Yes, you could have done a site-by-site upgrade: https://docs.splunk.com/Documentation/Splunk/7.3.0/Indexer/Upgradeacluster#Site-by-site_upgrade_for_...

And if you're already on a recent enough version (7.1.x or newer) you can even do a rolling upgrade of your indexers: https://docs.splunk.com/Documentation/Splunk/7.3.0/Indexer/Searchablerollingupgrade

ddrillic
Ultra Champion

@FrankVl, the documentation says -

-- When you initiate a rolling upgrade, you select a peer and take it offline. During the offline process, the master reassigns bucket primaries to other peers to retain the searchable state, and the peer completes any in-progress searches within a configurable timeout. See The fast offline process.

After the master shuts down the peer, you perform the software upgrade and bring the peer back online, at which point the peer rejoins the cluster. You repeat this process for each peer node until the rolling upgrade is complete.

Pretty amazing.

@FrankVl, we are not in maintenance mode, right?

0 Karma

FrankVl
Ultra Champion

Yes you are in maintenance mode during the rolling upgrade. See step 3: https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Searchablerollingupgrade#3._Initialize_r...

3. Initialize rolling upgrade
Run the following CLI command on the cluster master:
    splunk upgrade-init cluster-peers 
Or, send a POST request to the following endpoint:
    cluster/master/control/control/rolling_upgrade_init
This initializes the rolling upgrade and puts the cluster in maintenance mode.

ddrillic
Ultra Champion

Makes sense - thank you @FrankVl.

0 Karma

ddrillic
Ultra Champion

Gorgeous @FrankVl.

0 Karma

FrankVl
Ultra Champion

Please mark the answer as accepted if it solves your question.

0 Karma

ddrillic
Ultra Champion

From the SE -

-- Looks like you have your answers already. Site-by-site upgrades work in most cases.

There are caveats with certain breaking change type upgrades, but those are really rare, the most recent one I can remember is the SSL version change.

7.1+ does also allow for rolling upgrades as mentioned in the answers responses. This is likely your best path forward.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...