topic How do I remove a corrupted bucket in an Indexer Cluster environment? in Deployment Architecture

How do I remove a corrupted bucket in an Indexer Cluster environment?

daniel_splunk — Wed, 24 Oct 2018 04:26:27 GMT

One of my indexers crashed in a cluster environment and left a corrupted bucket. Search will return error when hitting that bucket like this:

[indexer1] idx=os Could not read event: cd=21145:261500. Results may be incomplete ! (logging only the first such error; enable DEBUG to see the rest)

Any command to remove/fix the corrupted bucket as I can't shutdown the indexer to run fsck right now?

Re: How do I remove a corrupted bucket in an Indexer Cluster environment?

daniel_splunk — Wed, 24 Oct 2018 04:31:23 GMT

From the message, the bucket number is 21145 (cd:21145:61500), you can run below search to locate the actual bucket.

| dbinspect index=os
| search bucketId = *21145*
| table bucketId, guId, splunk_server, index, state

Once you get the bucketId, run below REST API to remove it.

splunk _internal call /services/cluster/master/buckets/<bucketId>/remove_from_peer -method POST -post:peer <guId>

Re: How do I remove a corrupted bucket in an Indexer Cluster environment?

gregbo — Fri, 16 Nov 2018 09:52:08 GMT

How about a non-clustered bucket? Can I just delete it from the OS?