[ind2]
HomePath = $SPLUNK_DB/ind2/db
ColdPath = $SPLUNK_DB/ind2/colddb
ThawedPath = $SPLUNK_DB/ind2/thaweddb
MaxHotBuckets = 10
MaxDataSize = 10000
MaxWarmDBCount = 300
MaxTotalDataSizeMB = 50000
FrozenTimePeriodInSecs = 31536000
ColdToFrozenDir = $SPLUNK_DB/ind2/frozendb
In this config how long data will reside in each bucket
as per my understanding after 10gb data in hot ll roll to warm and when 300 bucket in warm ll full that ll roll to cold
or 10 hot bucket each bucket in hot ll take 10gb data than it ll roll to warm
is there aany one who can clarify???
maxwarmdbcount=300 means there can be maximum 300 warm buckets can exist on ind2 index HomePath.
Ideally all active bucket sizes can be maximum MaxDataSize which is 10GB on your ind2 index setting. Bucket sizes do not change while rolling to hot-->warm or warm-->cold. Only frozen buckets gets smaller because they contain only compressed raw-data.
Indexer restarts, wrong timestamp issues , low disk space may cause hot buckets to roll warm before reaching to MaxDataSize capacity. You can see many warm buckets smaller than MaxDataSize. For calculation you can consider them as MaxDataSize.
If this reply helps you an upvote is appreciated.
Hi @dall,
Addition to @richgalloway , since your low MaxTotalDataSizeMB setting, your warm db count cannot reach to 300. If there is no indexer restart 5 hot buckets will reach to MaxTotalDataSizeMB (50 GB) then go directly to frozen path (on every indexer restart, all hot buckets are rolled to warm). In order to keep data searchable for 365 days you should calculate the MaxTotalDataSizeMB using your daily ingestion rate.
You can check below documentation;
https://docs.splunk.com/Documentation/Splunk/8.1.1/Indexer/HowSplunkstoresindexes#How_data_ages
maxwarmdbcount=300 means once 300 bucket ll full than roll to next bucket
from here we need to ser size of warm bucket ya how to know size of this warm bucket??
All we know for sure is buckets will live for at least 31,536,000 seconds (365 days). That's a minimum value because the life of a buckets is based on the age of the newest event in that bucket. Without knowing the ingestion rate it's impossible to know how long a bucket will remain before it is frozen. For the same reason, we can't know how long the bucket will remain at each tier.
Of course, once frozen the bucket will stay there forever or until you thaw or delete it.