Deployment Architecture

How can we recover the empty bucket in the var/lib folder after a Splunk system crash?

tlam_splunk
Splunk Employee
Splunk Employee

After a Splunk crash, we are finding that there are a number of emptybucket-hot_v1_xxx in the /var/lib/... folder. Although we can find the new data coming and it can be searched, we are finding that some of the data is missing.

How could we recover the empty bucket ?

highsplunker
Contributor

Thanks a lot! It helped!

0 Karma

tlam_splunk
Splunk Employee
Splunk Employee

After the dirty shutdown, the bucket got corrupted and Splunk marked it for further investigation.

ls -laR emptybucket-hot_v1_xxx

Check that it has the journal.gz and necessary files...

Then do the following
1) Stop Splunk
2) make backup of that bucket
3) rename the bucket back to hot_v1_xxx
4) repair using fsck (and adding --include-hots) (save log output)
5) Start Splunk

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...