Deployment Architecture
Highlighted

How to set a time limit for buckets to remain warm

Path Finder

I know that there is a setting for how long a bucket will remain hot

maxHotSpanSecs

I also know there is a setting for how long before a bucket is either deleted or moved to frozen

frozenTimePeriodInSecs

But, what about time between warm and cold? Also, is there truly any difference between warm and cold other than name?

Tags (2)
0 Karma
Highlighted

Re: How to set a time limit for buckets to remain warm

Splunk Employee
Splunk Employee

In short, no.

There isn't much difference between warm and cold. The form of the bucket itself is identical in both states. (In hot, the bucket is still being written to.) Warm and cold buckets are static, and both are searchable. The warm to cold transition is the first boundary that lets us change partition. Consider a case where you have "fast" storage like an SSD, and "slow" storage like 5400 RPM rotating disks, and you want to keep stuff you search often on the SSDs. This is the most common use case for the warm to cold transition.

The usual way in trying to "keep buckets warm" is to enforce a count of buckets. Usually, this is a space constraint, rather than a time-based constraint, so people tune this with "if buckets can be up to 10G, and I have 300G of hot/warm storage, I can have a max of 30 warm buckets" (but don't forget to include the hot buckets, so really 27 warm buckets), and use the maxWarmDBCount parameter.

0 Karma