Rawdata may be corrupt

[profileaudio]

Hi anyone and everyone,

Please could somebody help.

I have been using Splunk for the past 2 and a half years.
I am using Splunk 5 and whenever I install a Splunk update over the existing Splunk 5, Splunk starts up as normal but after I perform a search, all the data will show until it gets to a point where it all vanishes and is replaced by the following.

Error in 'databasePartitionPolicy': Failed to read 1 event(s) from rawdata in bucket 'main~178~02C5891B-D87B-444E-9AEC-E9C8E3E45913'. Rawdata may be corrupt, see search.log

At this point I just reinstall the previous version as I need the search data.

As I know I am going to have to update it for good at some point can any one fix this corruption issue?

Kind regards,

Paul

[lukejadamec]

I've run into this before also, and there is a fix IF the actual data in the bucket is not corrupt. If the bucket raw data is truly corrupt, it cannot be fixed.

Here is a good place to read about fixing bad buckets:

http://wiki.splunk.com/Community:PostCrashFsckRepair

The repair routine never worked for me, so I use the rebuild instructions. However, sometimes those also fail for me, so modify the instructions a bit...

First try the instructions as written. If that fails try this on a copy of the bucket.

Remove all files inside the bucket except journal.gz - don't change the folder structure. Run rebuild on the bucket again, and it will be rebuilt from raw data. If that fails, then the data is likely unrecoverable.

[asmithe]

I have this same problem. Any answers?

Updated answer:

Without a service contract it is very difficult to get answers or a solution to this problem that dont include some data loss.

Ultimately, I had to track down the data buckets that had the corrupt data and remove them. Some of my SOS data is also corrupted and i never have gotten around to sorting out which data needs to be gone.

[khyoung7410]

I have this same problem. Any answers?