Deployment Architecture

How to restore frozen data when it looks my buckets have corrupted files?

guimilare
Communicator

Hi all.

I'm running Splunk 6.3.2 in Linux machines.

I have to restore some old data to Splunk.
I've followed the steps described at http://docs.splunk.com/Documentation/Splunk/6.3.2/Indexer/Restorearchiveddata but I'm facing an error that I can't solve.

I've copied the frozen bucket db_1438454243_1438385486_4557 to /var/splunk/defaultdb/thaweddb/
After that, I executed the following command:

/opt/splunk/bin/splunk rebuild /var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557 
USAGE: splunk rebuild <bucketPath> [<indexName>] [--ignore-read-error] [--no-log] The <indexName> parameter is ignored if provided. Please see 'splunk fsck' for more options.  This command is just a wrapper for 'splunk fsck'.

Redirecting to 'splunkd fsck' with args:
        repair --one-bucket --include-hots --bucket-path=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557
--log-to--splunkd-log ERROR JournalSlice - Error reading compressed journal while streaming: bad gzip header, provider=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata/journal.gz ERROR BucketBuilder - Error reading rawdata: Error reading compressed journal while streaming: bad gzip header, provider=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata/journal.gz WARN  Fsck - Repair (entire bucket) idx= bucket='/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557' failed: (entire bucket) Rebuild for bkt='/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557' failed: Error reading rawdata: Error reading compressed journal while streaming: bad gzip header, provider=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata/journal.gz Rebuilding bucket failed

Inside /var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata there are only 3 files: slicesv2.dat, slicemin.dat and journal.gz.

It looks like the buckets have corrupted files, since the command file in linux five me that the journal.gz file is data.
Is that correct or I'm missing something?

Thank in advance.
Best regards.

0 Karma

Masa
Splunk Employee
Splunk Employee

Yes, sounds like your bucket is corrupted.

How about "gunzip journal.gz" Does this work without error? Most likely this will return error, too.
Unfortunately, if that's the case, there is no tool available recover the bucket.

somesoni2
Revered Legend
0 Karma
Get Updates on the Splunk Community!

.conf25 Registration is OPEN!

Ready. Set. Splunk! Your favorite Splunk user event is back and better than ever. Get ready for more technical ...

Detecting Cross-Channel Fraud with Splunk

This article is the final installment in our three-part series exploring fraud detection techniques using ...

Splunk at Cisco Live 2025: Learning, Innovation, and a Little Bit of Mr. Brightside

Pack your bags (and maybe your dancing shoes)—Cisco Live is heading to San Diego, June 8–12, 2025, and Splunk ...