Splunk Search

How can I move an index in *.nix?

Anonymous
Not applicable

Hi,

I have index A stored on my systemdisk (i know), and I have made a new Index B on my datadisk.

How will I go forward with putting the IndexA events into IndexB, so I can delete IndexA.
Or just move the Index and restart Splunk?

What is the best way to fix this?

Is it possible to merge it or to move it?

Does anyone have experience with this?

System is running Red hat 7*

Thanks in advance for all help

0 Karma
1 Solution

nickhills
Ultra Champion

Adding a second answer - which I think targets your question - how to move an index.

Stop splunk
Copy the db folder from the old location to the new.
Assuming default paths this is probably something like
cp /opt/splunk/var/lib/splunk/my_old_index /opt/splunkdata/my_old_index
chown/chmod it so the splunk user owns it (probably not necessary in your case - but check)

Edit indexes.conf and update the paths for the old index to match the new paths. (backup this file a good idea!)
Note: depending on which app you were in when you created the index, determines which copy of indexes.conf you need to change.
Hint: it will be in <someapp>/local/indexes.conf - check in "search" and "launcher" apps - or grep for it!
Start splunk

Check the ui now has the new paths for your old index - and its searchable.
When you're sure, delete the data from the old path.

One final thing - if you installed to the default paths, and that happened to install onto your system volume, all your internal logs will be there too - you can move these in the same way - just double check each step.
Note: for the _internal indexes, you will find the original definitions in $SPLUNK_HOME/etc/system/default/indexes.conf - DONT change these. Instead copy the stanzas into $SPLUNK_HOME/etc/system/local/indexes.conf and make the changes there.

If my comment helps, please give it a thumbs up!

View solution in original post

woodcock
Esteemed Legend

Like this:

While splunk is running, rsync once.
Then rsync again.
Then stop splunk and rsync again.
Then modify indexes.conf to point to the new location.
Then start splunk.
If everything is OK, remove the old index directory and files.

skoelpin
SplunkTrust
SplunkTrust

I've done this before and it wasn't easy. If your on a standalone server then it should be much easier than in a distributed environment. TAKE A BACKUP BEFORE YOU START

0 Karma

nickhills
Ultra Champion

Adding a second answer - which I think targets your question - how to move an index.

Stop splunk
Copy the db folder from the old location to the new.
Assuming default paths this is probably something like
cp /opt/splunk/var/lib/splunk/my_old_index /opt/splunkdata/my_old_index
chown/chmod it so the splunk user owns it (probably not necessary in your case - but check)

Edit indexes.conf and update the paths for the old index to match the new paths. (backup this file a good idea!)
Note: depending on which app you were in when you created the index, determines which copy of indexes.conf you need to change.
Hint: it will be in <someapp>/local/indexes.conf - check in "search" and "launcher" apps - or grep for it!
Start splunk

Check the ui now has the new paths for your old index - and its searchable.
When you're sure, delete the data from the old path.

One final thing - if you installed to the default paths, and that happened to install onto your system volume, all your internal logs will be there too - you can move these in the same way - just double check each step.
Note: for the _internal indexes, you will find the original definitions in $SPLUNK_HOME/etc/system/default/indexes.conf - DONT change these. Instead copy the stanzas into $SPLUNK_HOME/etc/system/local/indexes.conf and make the changes there.

If my comment helps, please give it a thumbs up!

Anonymous
Not applicable

This is wonderful, and many thanks.
I wil get right on it and do some testing before i test this in production enviroment.

I wil update as soon as I have the results

0 Karma

Anonymous
Not applicable

The job is complete, it worked.

When I made the new index, there was no db folder.
So I copied the content of /oldindex/db/ into newindex/*
then it worked.

The fault I had done wrong was that I hade copied the folder DB, so the new index did not have the DB.
With some patient and help it worked:) Thnx:)

0 Karma

nickhills
Ultra Champion

I have posed some questions above, but in the meantime.

Generally data which you write to an index is not easy to 'move/copy/transplant' into another index.
The simple solution is simply to move the old index alongside the new one, and just accept that they are in two separate indexes (a mistake you'll only ever make once)

If you really want to 'remove' the old index, you could run a collect which would allow you to copy the event data into the new index.
There are two approaches - one which is '*free', the other '*costs' (*relative to your licence consumption)

if you have very few sourcetypes and are content with smushing them all down into one new stash sourcetype, you could do something like:

index=old * |collect index=new source=index_old addtime=true

You will end up with all of your old data under one source=old_index and sourcetype=stash - This wont consume any of your licence (hence 'free')

On the otherhand if you want to preserve source/sourcetype you can try:

index=old sourcetype=my_sourcetype_a |collect index=new source=source sourcetype=sourcetype addtime=true

However, this will count as a re-index, and will consume bytes from your licence (hence not free)

If you opt for the latter, I would do it one sourcetype at a time, and be prepared for some fun

If my comment helps, please give it a thumbs up!
0 Karma

Anonymous
Not applicable

Thank you for this, I wil use this as a emergency move.
Sine this wil count as a re-indexing incl sourcetypes.
Thanks for the reply

0 Karma

nickhills
Ultra Champion

Some Questions:

Do the indexes have different names?
Before you moved it, was data from both indexes searchable?
When you moved the 'old' index, did you update indexes.conf to reflect the new path?

If my comment helps, please give it a thumbs up!
0 Karma

Anonymous
Not applicable

I have done several things.
Yes, I have renamed it

Yes both indexes was created from GUI,
And searchable.

I did not update indexes.conf
The new name is already in indexes.conf

And I also put hot i one place and cold data on another disk.

And I copied so nothing is changed from the orginal index. (and it is safe)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...