Deployment Architecture

What happens to a bucket if all of the data within it is deleted?

Explorer

Just what the title says. If I delete a bunch of data from an index and some of the buckets are now effectively empty (E.g. all of them have been marked as deleted), what happens to the bucket? Is it removed or will it stick around until it is frozen?

0 Karma
1 Solution

Splunk Employee
Splunk Employee

It will stick around until the bucket is frozen. The bucket mover uses the epoch times in the bucket (directory) name and doesn't really inspect the contents of the bucket to determine whether it can be frozen or not.

View solution in original post

Splunk Employee
Splunk Employee

It will stick around until the bucket is frozen. The bucket mover uses the epoch times in the bucket (directory) name and doesn't really inspect the contents of the bucket to determine whether it can be frozen or not.

View solution in original post

Explorer

O.K. So, is there a way to determine if a bucket has only deleted entries? And, if so, can we force the bucket to be rolled to frozen? Or, is the file basically stuck there until it is frozen?

0 Karma

Splunk Employee
Splunk Employee

The short answer is no, buckets will honor the configured retention policy settings and there is no way to force a freeze unless you are (temporarily) reducing the index configuration. I am not aware of any way to safely determine whether a given bucket contains only deleted events.
The only way currently to physically delete events (buckets, really), is to run clean eventdata, which will wipe everything for the given index, so you'll have to come up with a plan to export/collect any events you do not want to delete.

What's your use case? Compliance? New EU regulations?

0 Karma

Explorer

Follow-up question: I did a little digging with dbinspect and noticed that some buckets can have an eventCount of zero, but the rawSize and/or sizeOnDiskMB is non-zero. This seems to indicate that these buckets have had all their records deleted, but are still taking up space on the system. Is that correct?

0 Karma

Explorer

It's more of a data cleanup issue. I was trying to avoid having to re-index a large chunk of data.

Thanks.

0 Karma

Splunk Employee
Splunk Employee

It depends what you did to delete the data: did you use the delete command (which does not actually remove anything from disk), or clean, or remove index?

If you have not already done so, I highly encourage you to read Remove indexes and indexed data in the Splunk Enterprise documentation.

0 Karma

Explorer

Just delete. I'm not trying to remove the entire index. I just wanted to know what would happen from a disk space perspective.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!