Splunk Enterprise

Which of these methods of adding new upgraded indexers to my cluster would be best?

Bomo2023
Explorer

I currently have a Splunk cluster that looks like this:

SplunkCentOS VersionSplunk Version
Master7.57.0.0
Forwarder7.5Universal Forwarder 6.6.3
Search Head6.57.0.0
Indexer 16.57.0.0
Indexer 26.57.0.0
Indexer 36.57.0.0
Indexer 46.57.0.0

 

I have 4 new, better servers that I want to build as indexers to replace the 4 indexers that I currently have (while moving from Splunk 7.0.0 to Splunk 8.x in the process).

It seems to me that I have two options:

Option 1
Create a new indexer cluster with the 4 new indexers, so that I have an old indexer cluster and a new indexer cluster.  Send new data to the new indexer cluster only and in time the data on the old indexer cluster will age off and I can retire that cluster. During this time I will configure the search head to search across both indexer clusters so that the old and new data is searchable.

Option 2
Add the new indexers to the existing indexer cluster, so that I have 4 old indexers and 4 new indexers in the same cluster. Allow the data to replicate across the 8 node cluster, and then retire the old indexers one by one. Leaving me with the 4 new indexers only.

In both scenarios above the old indexers will be CentOS 6.x and the new indexers will be CentOS 7.x, but as I understand it this won't be a problem.

Would anyone be able to offer any advice on which method would be best/easiest or point out any pitfalls or anything obvious that I may not have considered?

Labels (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Use Option 2.  Once the new indexers are added to the cluster, put the old indexers into detention.  Then run splunk offline --enforce-counts on one old indexer at a time.  The CM will make sure data from the old indexer is moved to the new indexers.  Wait until the old indexer stops before running the command on the next old indexer.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Use Option 2.  Once the new indexers are added to the cluster, put the old indexers into detention.  Then run splunk offline --enforce-counts on one old indexer at a time.  The CM will make sure data from the old indexer is moved to the new indexers.  Wait until the old indexer stops before running the command on the next old indexer.

---
If this reply helps you, Karma would be appreciated.

isoutamo
SplunkTrust
SplunkTrust

I also prefer the option 2. It will be much quicker way to get rid of old boxes. Also remember to rebalance buckets after you have removed old nodes from cluster.

Here https://community.splunk.com/t5/Splunk-Enterprise/Migration-of-Splunk-to-different-server-same-platf... is how I have done it as part of migrating multi site environment from one service provider to another.

r. Ismo

Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...