Getting Data In

Index has split into two. How can I merge the new index back into the old?

dtrelford
Path Finder

During the upgrade process for Splunk TA for Windows, the perfmon index location was moved. This resulted in two data locations - one with year of data and another with the last two weeks.

Searches for index=perfmon now only return results from events indexed in the new location.

I'd like to move the data from the new location back into the original location so everything is located within the single perfmon index & location. Is this possible?

0 Karma
1 Solution

gaurav_maniar
Builder

Hi,

It is possible by migrating db/buckets folders from the old index location to the new location.
Details with example
old index path - splunk_home/var/lib/splunk/perfmon
new index path - splunk_home/var/lib/splunk/perfmon_windows

Now both index path will be having coldb, thaweddb & db folder. These folders contain your index buckets.
The buckets folder name format - db_xxx_xxx_<id> or hot_v1_<id>.
Now move these bucket folders from old location to new location and while moving make sure that there are no duplicate ids in the bucket folder name.
Re-enumerate the IDs of the new buckets at the end of the existing range, or in a completely new range while copying to the new location.

make sure Splunk is stopped before the bucket movement

View solution in original post

0 Karma

gaurav_maniar
Builder

Hi,

It is possible by migrating db/buckets folders from the old index location to the new location.
Details with example
old index path - splunk_home/var/lib/splunk/perfmon
new index path - splunk_home/var/lib/splunk/perfmon_windows

Now both index path will be having coldb, thaweddb & db folder. These folders contain your index buckets.
The buckets folder name format - db_xxx_xxx_<id> or hot_v1_<id>.
Now move these bucket folders from old location to new location and while moving make sure that there are no duplicate ids in the bucket folder name.
Re-enumerate the IDs of the new buckets at the end of the existing range, or in a completely new range while copying to the new location.

make sure Splunk is stopped before the bucket movement

0 Karma

dtrelford
Path Finder

Would there be any issue with doing this in reverse? Moving new index buckets to old index location? The old index location format is the correct one.

0 Karma

dtrelford
Path Finder

Update: Moving the hot & warm buckets from the new index location back to the old AND coping the .manifest bucket info from the new to old .manifest worked.

0 Karma

mayurr98
Super Champion

you may need to use collect command.
refer this link
https://answers.splunk.com/answers/563872/merge-indices-to-one-index.html

0 Karma

dtrelford
Path Finder

The second index is 40Gb in size. Would the collect command be feasible to move this much data?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Using the Splunk Threat Research Team’s Latest Security Content

REGISTER HERE Tech Talk | Security Edition Did you know the Splunk Threat Research Team regularly releases ...

SplunkTrust | 2024 SplunkTrust Application Period is Open!

It's that time again, folks! That's right, the application/nomination period for the 2024 SplunkTrust is ...