How to migrate main index from current Splunk 6.2.5 instance to new 6.3.1 instance?

[tgord]

I have a situation where we're rolling out a new Splunk deployment on replacement hardware, and I want to start with a fresh install of Splunk, sans many of the no-longer-needed hacks that the previous instance is riddled with throughout its configuration. This means I will not be migrating over the entire $SPLUNK_HOME directory, so far only user information and saved searches, which I already know how to do.

Regarding the migration of indexes, I found the following thread:

https://answers.splunk.com/answers/3516/how-do-i-migrate-my-splunk-data-to-a-new-machine.html

The answers for which state that only the defaultdb folder need be copied over to successfully migrate the main index from one instance to another. However, this information was in reference to 4.0.1/4.1.3, and I am wondering - is it still accurate? The var/lib/splunk directories seem to differ quite considerably between our two instances at the moment, which is what gives me pause.

[jfeitosa_real]

I have a scenario, in which I have an indexer instance with 2TB in / opt, but it is 92% full.

What is the most efficient and safe way to migrate the indexes to a new instance or a new partition?

Thanks in advance.

[isoutamo]

Hi
Basically it depends are there need to e.g. refresh your hardware and/or os. If there is then the easiest way is to rsync /opt/splunk from old server and if it has installed from rpm/apt then install it over copied content.
You could find exact commands to do quite easily from answers by google.
r. Ismo

[jfeitosa_real]

Hi @isoutamo 

The issue is that the provisioned disk was made with Raid10, that is, it has 4 partitions of 1TB, but only available 2TB for / opt.

[root @ Hostname] # lsblk
NAME MAJ: MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259: 5 0 50G 0 disk
├─nvxy0n1p1 259: 6 0 1M 0 part
└─nvxy0n1p2 259: 7 0 50G 0 part /
nvxy1n1 259: 0 0 1000G 0 disk
└─md0 9: 0 0 2T 0 raid10 / opt
nvxy2n1 259: 1 0 1000G 0 disk
└─md0 9: 0 0 2T 0 raid10 / opt
nvxy3n1 259: 2 0 1000G 0 disk
└─md0 9: 0 0 2T 0 raid10 / opt
nvxy4n1 259: 3 0 1000G 0 disk
└─md0 9: 0 0 2T 0 raid10 / opt
nvxy5n1 259: 4 0 40G 0 disk

[root @ Hostname] # df -kh
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.6G 0 7.6G 0% / dev
tmpfs 7.6G 0 7.6G 0% / dev / shm
tmpfs 7.6G 377M 7.2G 5% / run
tmpfs 7.6G 0 7.6G 0% / sys / fs / cgroup
/ dev / nvxy0n1p2 50G 3.0G 48G 6% /
/ dev / md0 2.0T 1.7T 162G 92% / opt
tmpfs 1.6G 0 1.6G 0% / run / user / 1000

Thanks

[isoutamo]

I’m not so familiar with Linux sw raid (md), but I think that as you have raid 10 that means only 2TB max which has combined from those 4 x 1TB slices.
If/when you are building new environment from scratch I strongly suggest to use LVM for easier storage management.
r. Ismo

[jfeitosa_real]

Perfect @isoutamo 
That is the suggestion I made as well.
But to migrate to the new partition, I have doubts about the steps.Perfect.
That is the suggestion I made as well.
But to migrate to the new partition, I have doubts about the steps.

Thanks

[ChrisG]

The Managing Indexers and Clusters of Indexers manual has a topic that explains how to Move the index database.

[tgord]

This seems to describe how to change the filesystem location of the entire index db within a single instance. I have two separate and distinct instances, residing on different hardware, and I'd like to know if backing up the defaultdb folder on one instance and restoring it on the other is sufficient to migrate only the main index. Previous discussion seems to suggest this, but it's two major versions old.