Getting Data In

Index has split into two. How can I merge the new index back into the old?

dtrelford
Path Finder

During the upgrade process for Splunk TA for Windows, the perfmon index location was moved. This resulted in two data locations - one with year of data and another with the last two weeks.

Searches for index=perfmon now only return results from events indexed in the new location.

I'd like to move the data from the new location back into the original location so everything is located within the single perfmon index & location. Is this possible?

0 Karma
1 Solution

gaurav_maniar
Builder

Hi,

It is possible by migrating db/buckets folders from the old index location to the new location.
Details with example
old index path - splunk_home/var/lib/splunk/perfmon
new index path - splunk_home/var/lib/splunk/perfmon_windows

Now both index path will be having coldb, thaweddb & db folder. These folders contain your index buckets.
The buckets folder name format - db_xxx_xxx_<id> or hot_v1_<id>.
Now move these bucket folders from old location to new location and while moving make sure that there are no duplicate ids in the bucket folder name.
Re-enumerate the IDs of the new buckets at the end of the existing range, or in a completely new range while copying to the new location.

make sure Splunk is stopped before the bucket movement

View solution in original post

0 Karma

gaurav_maniar
Builder

Hi,

It is possible by migrating db/buckets folders from the old index location to the new location.
Details with example
old index path - splunk_home/var/lib/splunk/perfmon
new index path - splunk_home/var/lib/splunk/perfmon_windows

Now both index path will be having coldb, thaweddb & db folder. These folders contain your index buckets.
The buckets folder name format - db_xxx_xxx_<id> or hot_v1_<id>.
Now move these bucket folders from old location to new location and while moving make sure that there are no duplicate ids in the bucket folder name.
Re-enumerate the IDs of the new buckets at the end of the existing range, or in a completely new range while copying to the new location.

make sure Splunk is stopped before the bucket movement

0 Karma

dtrelford
Path Finder

Would there be any issue with doing this in reverse? Moving new index buckets to old index location? The old index location format is the correct one.

0 Karma

dtrelford
Path Finder

Update: Moving the hot & warm buckets from the new index location back to the old AND coping the .manifest bucket info from the new to old .manifest worked.

0 Karma

mayurr98
Super Champion

you may need to use collect command.
refer this link
https://answers.splunk.com/answers/563872/merge-indices-to-one-index.html

0 Karma

dtrelford
Path Finder

The second index is 40Gb in size. Would the collect command be feasible to move this much data?

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...