On the indexer in Splunk 6 getting this error message
10-25-2013 17:00:11.024 -0400 WARN IndexConfig - Max bucket size is larger than the index size limit. Please check your index configuration. idx=main; bucket size in MB (from maxDataSize) 10240, maxTotalDataSizeMB=1000
It does not state what index.
I checked the index configuration of the main application.
Max size (MB) of entire index : 50000
Max size (MB) of hot/warm/cold bucket is auto
Should i be worried about this message?
Yes, you can have a hot bucket larger than your maximum index size.
So the possible consequence is that a hot bucket will fill your storage, because it cannot be delete until it rotated to warm or cold.
So make sure than you have maxDataSize < maxTotalDataSizeMB
To confirm run a btool command on your indexes.conf
splunk cmd btool indexes list
and check for
for the explanation of the indexes parameters :
* The maximum size of an index (in MB).
* If an index grows larger than the maximum size, the oldest data is frozen.
* This paremeter only applies to hot, warm, and cold buckets. It does not apply to thawed buckets.
* Defaults to 500000.
* Highest legal value is 4294967295
* The maximum size in MB for a hot DB to reach before a roll to warm is triggered.
* Specifying "auto" or "autohighvolume" will cause Splunk to autotune this parameter (recommended).
* You should use "autohighvolume" for high-volume indexes (such as the main
index); otherwise, use "auto". A "high volume index" would typically be
considered one that gets over 10GB of data per day.
* Defaults to "auto", which sets the size to 750MB.
* "autohighvolume" sets the size to 10GB on 64-bit, and 1GB on 32-bit systems.
* Although the maximum value you can set this is 1048576 MB, which corresponds to 1 TB, a reasonable
number ranges anywhere from 100 to 50000. Before proceeding with any higher value, please seek
approval of Splunk Support.
* If you specify an invalid number or string, maxDataSize will be auto tuned.
* NOTE: The maximum size of your warm buckets may slightly exceed 'maxDataSize', due to post-processing and
timing issues with the rolling policy.
yes - it says that is in the
main index (this is the default index).
you should probably look in either of these places
note that for
The problem seems to be that you have configured a maximum size for the entire index to be 1GB, whereas the maximum size for any bucket within the index is at 10 GB. Thus - as soon as a hot bucket is rolled to warm, it will be frozen (most likely deleted).