Getting Data In

How can I trigger migration of buckets from Warm to Cold?

BunnyHop
Contributor

If the script to roll the hotDB to the warmDB is "| debug cmd=roll index=main", would there be one for rolling the warmDB to coldDB?

Tags (2)
1 Solution

BunnyHop
Contributor

This I will have to play with, but auto-generating an X amount of bucket (manual trigger of bucket generation per day using a script) and then setting that amount of bucket to move to cold once reached sounds like the solution.

View solution in original post

BunnyHop
Contributor

This I will have to play with, but auto-generating an X amount of bucket (manual trigger of bucket generation per day using a script) and then setting that amount of bucket to move to cold once reached sounds like the solution.

jrodman
Splunk Employee
Splunk Employee

Splunk index bucket management is usually not something you want to poke at manually. The hot-to-warm case is somewhat interesting because it comes up for backup purposes, and can be useful to force bucket sizing in time or in space on your own terms instead of with Splunk's pre-packaged logic. The warm-to-cold case is only interesting when dealing with multiple datastores (multiple filesystems).

However, this does become a point of interest when first setting splunk up, in order to validate behavior and operation. There's no easy way to force it, so the general method is to simply constrict the allowed number of warm buckets to force some to reach cold. In indexes.conf (generally set up in etc/system/local/indexes.conf) you can set the maxWarmDBCount on a index-by-index basis.

maxWarmDBCount = <integer>
* The maximum number of warm DB_N_N_N directories.
* All warm DBs are in the <homePath> for the index. 
* Warm DBs are kept in open state.
* Defaults to 300.

This means you can temporarily configure your main index (say in the initial setup case) or you could configure a test index to try things with.

Get Updates on the Splunk Community!

Splunk Classroom Chronicles: Training Tales and Testimonials

Welcome to the "Splunk Classroom Chronicles" series, created to help curious, career-minded learners get ...

Access Tokens Page - New & Improved

Splunk Observability Cloud recently launched an improved design for the access tokens page for better ...

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

&#x1f342; Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...