【emergency】I can not migrate indexed data

[kawashita_t]

I want to transfer indexed data to another server.

I did the following method to realize it.

Migration of index "main_export".
１、Copy the db folder under the main_export folder of the server before migration.
２、Create index of main_export at destination server.
３、Stopping SPlunk on destination server.
４、Copy the contents copied earlier to the db folder under main_export of the destination server.
５、Start Splunk of the destination server.

However, with this method the index was not successfully migrated.
This is the error content at this time.


12-29-2017 10:52:07.837 +0900 INFO DatabaseDirectoryManager - Getting size on disk: Unable to get size on disk for bucket id=main_export~137~8804D8D6-CCD6-43A2-8F60-4949589D231D path="C:\Program Files\Splunk\var\lib\splunk\main_export\db\db_1509511194_1509506867_137" (This is usually harmless as we may be racing with a rename in BucketMover or the S2SFileReceiver thread, which should be obvious in log file; the previous WARN message about this path can safely be ignored.) caller=getCumulativeSizeForPaths
12-29-2017 10:52:07.837 +0900 INFO DatabaseDirectoryManager - Getting size on disk: Unable to get size on disk for bucket id=main_export~138~8804D8D6-CCD6-43A2-8F60-4949589D231D path="C:\Program Files\Splunk\var\lib\splunk\main_export\db\db_1509508148_1509503968_138" (This is usually harmless as we may be racing with a rename in BucketMover or the S2SFileReceiver thread, which should be obvious in log file; the previous WARN message about this path can safely be ignored.) caller=getCumulativeSizeForPaths


I would like to tell you how to deal with it.
The SPlunk you use is 6.6 series.

if you'd kindly teach me.

[Elsurion]

It looks like you didn't copy the index, just a few files.
And let's create splunk the index folder structure for you by adding the index to the index.conf an restart splunk.
When you do a move/copy of an index, copy everything, including colddb, db, hot, etc.
like the following commando

cp -rp ~/var/lib/splunk/<your_index> <path>/<to>/<your>/<storage>
or do a drag an drop on your windows system

The normal index structure looks like the follow

me@myserver:~/var/lib/splunk/elsurion> tree
.
├── colddb
├── datamodel_summary
├── db
│   │   ...
│   ├── db_1484138704_1484138344_99
│   │   ├── 1484138704-1484138344-5030796207003270706.tsidx
│   │   ├── bloomfilter
│   │   ├── bucket_info.csv
│   │   ├── Hosts.data
│   │   ├── optimize.result
│   │   ├── rawdata
│   │   │   ├── ournal.gz
│   │   │   ├── slicemin.dat
│   │   │   └── slicesv2.dat
│   │   ├── Sources.data
│   │   ├── SourceTypes.data
│   │   └── Strings.data
│   │   ...
│   ├── GlobalMetaData
│   └── hot_v1_143
│       ├── 1514495803-1514448509-617512707267247429.tsidx
│       ├── 1514532124-1514520088-9040980535112679088.tsidx
│       ├── 1514532128-1514532128-9049526622478475961.tsidx
│       ├── 1514532138-1514532138-9092459381857281969.tsidx
│       ├── 1514532139-1514532079-9096760486661566399.tsidx
│       ├── 1514532148-1514532148-9135411910970549314.tsidx
│       ├── 1514588396-1514495805-12721369008098433568.tsidx
│       ├── bucket_info.csv
│       ├── Hosts.data
│       ├── rawdata
│       │   ├── 6163361
│       │   ├── journal.gz
│       │   └── slicesv2.dat
│       ├── Sources.data
│       ├── SourceTypes.data
│       ├── splunk-need-optimize.dat
│       └── Strings.data
└── thaweddb

Also you have to pay attention to the dat file

me@myserver:~/var/lib/splunk> ls -al
...
drwx------  6 splunk splunk    71  5. Feb 2016  elsurion
-rw-------  1 splunk splunk     3 28. Dez 09:08 elsurion.dat

In this dat file the indexer keeps records of the actual bucket. Edit it or just delete it (splunk adds it again with correct value when restarting) when creating an empty index before copy your old index over. Or copy the file with your index.

[kunalmao]

there will also be a folder with index name in splunkdb in the location where you are storing your indexed data, you can check that in indexes.conf. copy both the files and then carry out the steps which you have mentioned.