Splunk Enterprise

index bucket

dall
Path Finder

 [ind2]

HomePath = $SPLUNK_DB/ind2/db

ColdPath = $SPLUNK_DB/ind2/colddb

ThawedPath = $SPLUNK_DB/ind2/thaweddb

MaxHotBuckets = 10

MaxDataSize = 10000

MaxWarmDBCount = 300

MaxTotalDataSizeMB = 50000

FrozenTimePeriodInSecs = 31536000 

 ColdToFrozenDir = $SPLUNK_DB/ind2/frozendb

 

In this config how long data will reside in each bucket

as per my understanding after 10gb data in hot ll roll to warm and when 300 bucket in warm ll full that ll roll to cold 

or 10 hot bucket each bucket in hot ll take 10gb data than it ll roll to warm

is there aany one who can clarify??? 

Labels (1)
0 Karma

scelikok
SplunkTrust
SplunkTrust

maxwarmdbcount=300 means there can be maximum 300 warm buckets can exist on ind2 index HomePath.

Ideally all active bucket sizes can be maximum MaxDataSize which is 10GB on your ind2 index setting. Bucket sizes do not change while rolling to hot-->warm or warm-->cold. Only frozen buckets gets smaller because they contain only compressed raw-data.

Indexer restarts, wrong timestamp issues , low disk space may cause hot buckets to roll warm before reaching to MaxDataSize capacity. You can see many warm buckets smaller than MaxDataSize.  For calculation you can consider them as MaxDataSize.

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.

scelikok
SplunkTrust
SplunkTrust

Hi @dall,

Addition to @richgalloway , since your low MaxTotalDataSizeMB setting, your warm db count cannot reach to 300. If there is no indexer restart 5 hot buckets will reach to MaxTotalDataSizeMB (50 GB) then go directly to frozen path (on every  indexer restart, all hot buckets are rolled to warm).  In order to keep data searchable for 365 days you should calculate the MaxTotalDataSizeMB  using your daily ingestion rate. 

You can check below documentation;

https://docs.splunk.com/Documentation/Splunk/8.1.1/Indexer/HowSplunkstoresindexes#How_data_ages 

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

dall
Path Finder

maxwarmdbcount=300 means once 300 bucket ll full than roll to next bucket

from here we need to ser size of warm bucket ya how to know size of this warm bucket??

0 Karma

richgalloway
SplunkTrust
SplunkTrust

All we know for sure is buckets will live for at least 31,536,000 seconds (365 days).  That's a minimum value because the life of a buckets is based on the age of the newest event in that bucket.  Without knowing the ingestion rate it's impossible to know how long a bucket will remain before it is frozen.  For the same reason, we can't know how long the bucket will remain at each tier.

Of course, once frozen the bucket will stay there forever or until you thaw or delete it.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...