Getting Data In

How to move data from one peer node to another peer in an indexer clustering environment?

dpraveen88
Explorer

I have 3 indexers in cluster master. (Indexer 1, indexer2 and indexer3)
I need to stop indexer2 and indexer3 permanently.
To Stop the indexers in the cluster, I use "offline" command. It stops the indexer offline. Now I need to move the buckets (data) from indexer2 ,3 to indexer 1.

Please help me the process steps to move buckets to existing indexers.

Thanks!

0 Karma
1 Solution

lguinn2
Legend

NO!! You cannot simply move buckets from one indexer to another. You shouldn't do this in general and you definitely shouldn't do it on an indexer cluster. Chances are high that you will corrupt all of the data. But there is a way to have Splunk do this for you, and it is pretty simple. First question: did you use

splunk offline --enforce-counts

or just

splunk offline

to take indexers offline? If you used "enforce-counts" AND you waited for each indexer to fully stop, then: congratulations! You are done!! The cluster master automatically made copies of all necessary data to the surviving indexer.

First, without enforce-counts, "offline" only means that the indexer is only going to be offline for a restart. That is not the case here. So, start the indexers (indexer2 and indexer3) again, and do the offline with enforce-counts. It may take a long time for each indexer to fully stop, but you need to wait it out.

View solution in original post

lguinn2
Legend

NO!! You cannot simply move buckets from one indexer to another. You shouldn't do this in general and you definitely shouldn't do it on an indexer cluster. Chances are high that you will corrupt all of the data. But there is a way to have Splunk do this for you, and it is pretty simple. First question: did you use

splunk offline --enforce-counts

or just

splunk offline

to take indexers offline? If you used "enforce-counts" AND you waited for each indexer to fully stop, then: congratulations! You are done!! The cluster master automatically made copies of all necessary data to the surviving indexer.

First, without enforce-counts, "offline" only means that the indexer is only going to be offline for a restart. That is not the case here. So, start the indexers (indexer2 and indexer3) again, and do the offline with enforce-counts. It may take a long time for each indexer to fully stop, but you need to wait it out.

dpraveen88
Explorer

Thanks for responding for you valuable suggestion. I used already this command "splunk offline --enforce-counts". so far i stopped the indexer3 permanently. After that whatever the old data is available in indexer3, i need to move from indexer3 to indexer1.

0 Karma

masonmorales
Influencer

If you have the storage, just increase the search factor and replication factor to 3 on the cluster master and let the buckets replicate on their own. Then, you can just remove indexer 2 and indexer 3 from the cluster.

0 Karma

hunderliggur
Path Finder

To reduce a cluster from 3 to 1:
Set search factor 1 replication factor 2.

Let the cluster stabilize.

Remove indexer 3 with a controlled stop to remove it from the cluster (splunk offline --enforce-counts).
Let the cluster stabilize.
Remove indexer 2 with a controlled stop to remove it from the cluster (splunk offline --enforce-counts).
Let the cluster stabilize.
You are all done.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...