topic Re: Critical Bucket size and range in Getting Data In

Critical Bucket size and range

jamie1 — Tue, 16 Jan 2024 10:24:27 GMT

Hi There,

I have noticed that the cloud monitoring console is reporting a critical bucket. I only have one and have attached a screenshot. The small % is 100.

Unfortunately, I am not certain as to what this really means and whether it is something to worry about or not.

Any help would be appreciated,

Jamie

Re: Critical Bucket size and range

colbym — Mon, 14 Apr 2025 19:57:19 GMT

I have the same question

Re: Critical Bucket size and range

richgalloway — Mon, 14 Apr 2025 20:42:12 GMT

This is an indication of inefficient bucket use, meaning buckets roll `before they fill up. This can happen when indexers restart often, but in this case I suspect it's just a matter of the main index getting very few events before maxHotSpecSecs is reached and the bucket rolls to warm.

The answer for buckets that are known to contain few events is to set maxDataSize to a value that makes the bucket at least 50% full before it rolls. The default bucket size is 750MB. The dbinspect command can tell you the current size of buckets to give you an idea of how to set maxDataSize.

Best Practice is to not use the main index at all. All incoming data should go into a custom index, leaving main empty (and not needing to roll).

Re: Critical Bucket size and range

isoutamo — Tue, 15 Apr 2025 20:45:40 GMT

Another reason could be that your events contains timestamps from very far away each other. This also leads that buckets will close earlier than those are full.

There should be some indications for reason in _internal logs or even some CMC -> Indexing -> Data quality.