Deployment Architecture

How to move an index to new indexer cluster?

ankithreddy777
Contributor

Hi,
I have an Index,

home path= /data/splunk/indexes/home/index_name/db
cold path = /data/splunk/indexes/cold/index_name/db
thawed path= $Splunk_DB/index_name/
$Splunk_DB=/opt/splunk/var/lib/splunk

I have to move this index to new indexer cluster. Usually in the documentation it is given to move data in Splunk_DB to new location when you want to move indexes. But my data locations are different (not in Splunk_DB). May I know more about what is the procedure? Moreover I see .dat files in $SPLUNK_DB. what is the significance of those files?

Description from other duplicate question:
How to move index to new cluster where data is in one location(/data/splunk/indexes/.../index_name/db) and .dat files and thawed path in other location(/opt/splunk/var/lib/splunk).

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi TStrauch,
in https://docs.splunk.com/Documentation/Splunk/6.5.1/Indexer/Moveanindex is described how to move an index from a location to another.

You said that you want to move an index from a non clustered indexer to a cluster, did I correctly undestand?
remember that an indexer cluster replies only the new events, so if you have old events in your index you cannot copy index files in the new location (they aren't replied)!
I had a similar problem and I solved in this way:

  • I stopped Splunk;
  • I created a new index on cluster (using Master Node) with a different name in a different location;
  • I copied the old index files in all the cluster peers;
  • I redirected all the inputs into the new clustered index;
  • I restarted Splunk;
  • I created an eventtype (index=old_index OR index=new_index) and I used it in my searches instead of index=old_index.

Bye.
Giuseppe

0 Karma

somesoni2
Revered Legend

How are you planning to store the data in new cluster, will the homePath/coldPath and thawedPath will be different there as well? Actually in either case,you move data from old homePath/coldPath to new cluster's homePath/coldPath and data from old thawedPath to new cluster's thawedPath.

0 Karma

ankithreddy777
Contributor

ok got it. what I should do with the .dat files present in $SPLUNK_DB directory. should I move them as well to the SPLUNK_DB directory in new server

0 Karma

bsellapi
New Member

Hey ankith

How you solved this. We are having similar schenario, need to move the indexed data to new environment.
Please advise.

Thanks
Bala

0 Karma

somesoni2
Revered Legend

I don't think you need that. I believe it just keeps track of next hot bucket id to use.

0 Karma

TStrauch
Communicator

Hi,

just give me a response if i understand something wrong.

Your main problem is that you need to change the "Splunk_DB" variable am i right?

To change the "Splunk_DB" variable just du this.

Stop Splunk
Unset the Splunk_DB variable by "unset SPLUNK_DB"
Then go to "$SPLUNK_HOME/etc/splunk_launch.conf" and change the "SPLUNK_DB" variable to the path of your choice.
Start Splunk

I think there should be no problem by migrating your indexes as described in Docs. The described way in Docs should work for this scenario.

For the .dat files im not 100% sure but i think they hold the next bucket_id for the index.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...