Splunk Enterprise

Which of these methods of adding new upgraded indexers to my cluster would be best?

Bomo2023
Explorer

I currently have a Splunk cluster that looks like this:

SplunkCentOS VersionSplunk Version
Master7.57.0.0
Forwarder7.5Universal Forwarder 6.6.3
Search Head6.57.0.0
Indexer 16.57.0.0
Indexer 26.57.0.0
Indexer 36.57.0.0
Indexer 46.57.0.0

 

I have 4 new, better servers that I want to build as indexers to replace the 4 indexers that I currently have (while moving from Splunk 7.0.0 to Splunk 8.x in the process).

It seems to me that I have two options:

Option 1
Create a new indexer cluster with the 4 new indexers, so that I have an old indexer cluster and a new indexer cluster.  Send new data to the new indexer cluster only and in time the data on the old indexer cluster will age off and I can retire that cluster. During this time I will configure the search head to search across both indexer clusters so that the old and new data is searchable.

Option 2
Add the new indexers to the existing indexer cluster, so that I have 4 old indexers and 4 new indexers in the same cluster. Allow the data to replicate across the 8 node cluster, and then retire the old indexers one by one. Leaving me with the 4 new indexers only.

In both scenarios above the old indexers will be CentOS 6.x and the new indexers will be CentOS 7.x, but as I understand it this won't be a problem.

Would anyone be able to offer any advice on which method would be best/easiest or point out any pitfalls or anything obvious that I may not have considered?

Labels (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Use Option 2.  Once the new indexers are added to the cluster, put the old indexers into detention.  Then run splunk offline --enforce-counts on one old indexer at a time.  The CM will make sure data from the old indexer is moved to the new indexers.  Wait until the old indexer stops before running the command on the next old indexer.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Use Option 2.  Once the new indexers are added to the cluster, put the old indexers into detention.  Then run splunk offline --enforce-counts on one old indexer at a time.  The CM will make sure data from the old indexer is moved to the new indexers.  Wait until the old indexer stops before running the command on the next old indexer.

---
If this reply helps you, Karma would be appreciated.

isoutamo
SplunkTrust
SplunkTrust

I also prefer the option 2. It will be much quicker way to get rid of old boxes. Also remember to rebalance buckets after you have removed old nodes from cluster.

Here https://community.splunk.com/t5/Splunk-Enterprise/Migration-of-Splunk-to-different-server-same-platf... is how I have done it as part of migrating multi site environment from one service provider to another.

r. Ismo

Get Updates on the Splunk Community!

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

[Coming Soon] Splunk Observability Cloud - Enhanced navigation with a modern look and ...

We are excited to introduce our enhanced UI that brings together AppDynamics and Splunk Observability. This is ...