How to migrate data in an indexer cluster to a new indexer cluster instance?

[amoldesai]

Hi,

We have an application running in production on Splunk Enterprise version 6.5.1. The data available in the production indexer cluster needs to migrated to a new instance of indexer clusters. These new indexers are running on 6.5.1 too.

Can you please suggest how the indexed data can be migrated to the new indexer clusters?

Thanks a lot.

[behlkush]

I am in a similar situation and would like to know what you finally selected and went ahead with. Did you go ahead with what woodcock suggested.

[woodcock]

We typically do not migrate data. It is possible but it is easy to make a mistake and then you have a big mess. Instead, what we do is move the new indexers to production including updating them as search peers. Then change outputs.conf everywhere to only point to the new indexers. Because nothing new is coming to the old indexers, you just wait for the data to age out naturally and then decommission them.

[pedroponchio]

What about if we have to migrate to a new hardware due to warranty or hardware issues?
Do you think this approach would work for an indexer cluster single-site?

• Start a rsync from a peer A on the cluster to a new host B, warm and cold buckets
• Put CM on maintenance mode
• Remove peer A from cluster: splunk offline --./splunk offline --enforce-counts
• Final rsync job to sync the latest changes on hot and cold volumes from A to host B
• Have a copy of the clustered indexes.conf on the etc/system/local/ of host B
• Start splunk on host B
• Add host B to the cluster and restart host B
• Push new outputs.conf to all UFs and HFs with the new address of the peer B, and to remove peer A
• If all goes well, then we permanently remove the old peer A: splunk remove cluster-peers -peers
• Repeat until all peers are removed from Cluster and moved to the new hosts. Thanks!

[woodcock]

You will have to restart the CM after every indexer swap out but that should work fine.

[lfedak_splunk]

Hey @pedroponchio, since this original post is from February -- please create a new Answers post. You can include a link to this post to reference what you've already researched and then specify the difference in your environment and conditions.