Generate hashes for buckets that were created before data integrity control was enabled

[jniedrauer]

After enabling data integrity control for an index, I cannot find a way to generate hashes for existing buckets.

`./splunk generate-hash-files` returns `Error: Cannot generate hash files for the bucket with path=/bucket/path, Reason=Journal has no hashes.`

Rebuilding the buckets with:

`./splunk fsck repair` does not generate hashes for the old buckets.

Is this possible? Or are all buckets that were created before this option was enabled impossible to validate, even after the feature has been enabled?

[jniedrauer]

Interesting. So to be clear, in your experience, there is no way to retroactively create hashes for a journal?

[hmallett]

It's many years since I had to deal with the details of enableDataIntegrityControl, and the detail in the documentation misses some of the detail that I'm sure I learned years ago.

My memory is that when you enable data integrity control the hashes of the data are written with each slice in the data (this isn't mentioned in the documentation), and a copy of the hashes are stored in files in the raw data for the bucket (this is mentioned in the documentation).

When you run the check-integrity command the hashes from within the data are verified against the hash files. When you run the generate-hash-files command it will generate the hash files from the hashes stored in each slice. It won't create the hashes against each slice. That is only done when a slice is written.

As I say, it was many years since I had to deal with the detail of this, so things may have changed, or I may have misremembered, but that would explain why you don't get the results you're hoping for.