Splunk Enterprise

Indexes bucket settings

Abilan1
Path Finder

Hi ,

I see that "Max size (MB) of hot/warm/cold bucket = auto" under my one of index, I would like to know about this setting, like how long data will be in hot/warm/cold buckets? also If we have this setting for index, then frozenTimePeriodInSecs will not work?

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

This is probably the setting "maxDataSize " for the bucket size (auto = 750MB/ bucket)l.
This is not the size of the index, but the size of the unit of storages (buckets)

see in the specifications : http://docs.splunk.com/Documentation/Splunk/latest/admin/Indexesconf

maxDataSize = <positive integer>|auto|auto_high_volume
* The maximum size in MB for a hot DB to reach before a roll to warm is triggered.
* Specifying "auto" or "auto_high_volume" will cause Splunk to autotune this parameter (recommended).
* You should use "auto_high_volume" for high-volume indexes (such as the main index); otherwise, use "auto".  A "high volume index" would typically be considered one that gets over 10GB of data per day.
* Defaults to "auto", which sets the size to 750MB.
* "auto_high_volume" sets the size to 10GB on 64-bit, and 1GB on 32-bit  systems.
* Although the maximum value you can set this is 1048576 MB, which   corresponds to 1 TB, a reasonable number ranges anywhere from 100 to    50000.  Before proceeding with any higher value, please seek approval of    Splunk Support.
* If you specify an invalid number or string, maxDataSize will be auto    tuned.
* NOTE: The maximum size of your warm buckets may slightly exceed    'maxDataSize', due to post-processing and timing issues with the rolling   policy.

if you are looking for the index size limits, look at "maxTotalDataSizeMB" and the volumes definitions.

Abilan1
Path Finder

Hi ,

It means hot bucket size is 750 MB right? And what about Warm and cold bucket size? And where I can find this settings?

0 Karma

yannK
Splunk Employee
Splunk Employee

The hot buckets are read/write, the warm and cold are read only. Therefore when the buckets rotate
hot -> warm -> cold, their size stays the same, and is defined by the size when they were hot.

0 Karma

Abilan1
Path Finder

So it means all hot, warm, cold buckets will be only 750 MB? After 750 MB filled in Cold db what will happen? data will be deleted?

0 Karma

yannK
Splunk Employee
Splunk Employee

No, the bucket is size is not the index size, a bucket is an unit of storage time based, with span, number and size limits.
An index is composed of many buckets in different states.
they are also additional limits on the number of hot buckets, and warm buckets, but not cold buckets, for storage reasons.

example :
if for your index "myindex" maxTotalDataSizeMB = 100GB
then you will have inside buckets of chunk of data between a few MB to 750MB
to if you have a very streamline ingestion (and buckets of 750MB all) about 136 buckets.

(100*1024/750)

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...