topic Re: Migrating Indexed data to different splunk instance in Installation

Migrating Indexed data to different splunk instance

Gowtham0809 — Sat, 06 Jun 2020 15:00:00 GMT

Hello

Is it possible to migrate indexed data from one Splunk instance to another splunk instance.

I have a few indexes which have been holding data for a few years and now I need to move this to a different spunk instance. New Splunk instance completely new environment without any of the old indexes created in it

Thanks,

Re: Migrating Indexed data to different splunk instance

gcusello — Thu, 21 Nov 2019 10:22:42 GMT

Hi @Gowtham0809,
yes it's possible to migrate data from a Splunk instance to another.
There are two situations:

If in the same index of the new instance you haven't data;
If in the same index of the new instance you already have data.

In the first case, you can follow the instructions at https://docs.splunk.com/Documentation/Splunk/8.0.0/Indexer/Moveanindex and move indexes from the old instance to the new one.

In the second case, the easier approach is to export data in row format (e.g. using a search index=my_index and export data)
and reindexing them in the new instance.
Put attention when you export data that you are exporting all the data ( https://docs.splunk.com/Documentation/Splunk/8.0.0/Search/ExportdatausingSplunkWeb ).

Obviously you'll exceed your license for a day, but usually it isn't a problem, anyway put attention if you have other exceedings in the last 30 days to avoid license violations.

Ciao.
Giuseppe

Re: Migrating Indexed data to different splunk instance

woodcock — Thu, 21 Nov 2019 15:30:42 GMT

It is easy if the new system has no data; just rsync the entire directory structures for the indexed data over.

If the old system has data for the same index values, you need to understand the meaning of the <indexname>.dat file and you may need to modify bucket IDs. A bucket looks like this db_1571769855_1571338901_193 The _193 is the bucket ID. This is arbitrary (except for the value in <indexname>.dat) and must be unique within any index (across both warm and cold directories). If you have two ..._123 buckets, one of them will have to be renamed. The trick is that the value in <indexname>.dat is the next available incrementalbucket ID` value for that index. You need to make sure that as you are reconciling conflicts and consuming higher numbers, that you also bump up that number.