rawdata restore possible?

[gregwilliams]

I am attempting to recover from a hard crash, through no fault of Splunk's. Is it possible to unzip /rawdata/journal.gz and import that raw data back into a new index? I had multiple indexes, so there are are multiple folders with rawdata/journal.gz in them. Just looking to unzip them all into 1 index.

[jrodman]

Adam has the good answer here, but to be explicit: you cannot uncompress journal.gz and re-compress it sanely. slices.dat and journal.gz must agree and both must exist.

If you somehow encounter a corruption mid-journal.gz and need to perform partial recovery because it's hugely important data and you have no backups (!!) and are willing to spend many hours on the problem, get in touch directly (username) or via support.

Generally Splunk shouldn't be capable of generating such a problem as we don't modify these files, so any such problem would have to be OS or hardware faults.

[ahattrell_splun]

Found this question whilst searching for the same answer - thought I'd give the answer now I've worked it out.

In my case I had a journal.gz but had no idea where it came from originally.

i) Create a directory for the bucket:

• mkdir /tmp/newBucket
• mkdir /tmp/newBucket/rawdata

ii) Copy in the journal.gz

• cp journal.gz /tmp/newBucket/rawdata/

iii) Now run rebuild

• ./splunk rebuild /tmp/newBucket

iv) Determine the bucket name. We can get this from the tsidx files.

• ls /tmp/newBucket/*.tsidx

Gives

• 1351216920-1351213200-1612246429890249572.tsidx
• 1353460020-1351213200-999025018140623716.tsidx

The format is db_max_min_uniqid where

• max is the highest of the first number in the tsidx file - in this case 1353460020
• min is the lowest of the second number in the tsidx filename - in this case 1351213200
• uniqid is a unique integer that is not currently used in my target index. I used 1.

Thus we get:

• mv /tmp/newBucket /tmp/db_1353460020_1351213200_1

v) Copy this directory into the target indexes thaweddb directory. I created an index called recovered so:

• mv /tmp/db_1353460020_1351213200_1 /Applications/splunk/var/lib/splunk/recovered/thaweddb/

vi) Run:

• ./splunk _internal call /data/indexes/main/rebuild-metadata-and-manifests

Wait a short while and then run a search on your new index.

[melonman]

I was trying to follow this step with Splunk6.0.2, but it did not work. Any chance since this question answered?