Deployment Architecture

How can we recover the empty bucket in the var/lib folder after a Splunk system crash?

tlam_splunk
Splunk Employee
Splunk Employee

After a Splunk crash, we are finding that there are a number of emptybucket-hot_v1_xxx in the /var/lib/... folder. Although we can find the new data coming and it can be searched, we are finding that some of the data is missing.

How could we recover the empty bucket ?

highsplunker
Contributor

Thanks a lot! It helped!

0 Karma

tlam_splunk
Splunk Employee
Splunk Employee

After the dirty shutdown, the bucket got corrupted and Splunk marked it for further investigation.

ls -laR emptybucket-hot_v1_xxx

Check that it has the journal.gz and necessary files...

Then do the following
1) Stop Splunk
2) make backup of that bucket
3) rename the bucket back to hot_v1_xxx
4) repair using fsck (and adding --include-hots) (save log output)
5) Start Splunk

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...