Deployment Architecture

How to rollback buckets from cold to warm?

joxley
Path Finder

I have recently upgraded my indexer to have two sets of drives. SSDs are mounted on /fast and spinning rust is sitting on /cold. I have configured the indexes to have homePath on the /fast partition and coldPath on the /cold partition.

Because this is a new setup, there is a lot of data in the cold indexes already. I'd like to un-roll the indexes so everything is on the /fast partition and only roll over when it fills up.

1 Solution

lguinn2
Legend

Here are the steps:

  1. Stop Splunk.
  2. Make a backup of the indexes, to be safe.
  3. Copy all the buckets in the cold directory to the new home directory for each index.
  4. Remove the buckets from the cold directory.
  5. Check the settings in all index.conf files - the default number of warm buckets is 300; if you have not specified a larger number, then you should! For example: maxWarmDBCount = 10000 or a number large enough to fill your /fast partition. Also, you should be using volumes to manage the size of the home directory (hot + warm) to avoid completely filling the disk.
  6. Start Splunk.

Splunk will roll the warm buckets to cold (again) when it either hits the maxWarmDBCount or the volume size for hot/warm.

View solution in original post

lguinn2
Legend

Here are the steps:

  1. Stop Splunk.
  2. Make a backup of the indexes, to be safe.
  3. Copy all the buckets in the cold directory to the new home directory for each index.
  4. Remove the buckets from the cold directory.
  5. Check the settings in all index.conf files - the default number of warm buckets is 300; if you have not specified a larger number, then you should! For example: maxWarmDBCount = 10000 or a number large enough to fill your /fast partition. Also, you should be using volumes to manage the size of the home directory (hot + warm) to avoid completely filling the disk.
  6. Start Splunk.

Splunk will roll the warm buckets to cold (again) when it either hits the maxWarmDBCount or the volume size for hot/warm.

Lucas_K
Motivator

So the deletion of .bucketmanifest is not required anymore? ie. manifests are now checked against existing files upon start up "every time"? I hadn't checked if this was still required in any v6 version.

Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...