Getting Data In

re-balance disk space on indexers?

toddblake
Explorer

We have an environment that had 9 indexers, and we just added 6 more. Our old indexers were all getting up to around 88% capacity, and our new ones are of course at 0. Is it possible to shuffle buckets around to even these out? I've read articles on moving entire indexes between indexers, but I'm wondering if there's a way to basically move a percentage of the buckets in each of the old 9 to the new so they all even out at 52%(by my math).

Tags (1)

dwaddle
SplunkTrust
SplunkTrust

Yes, you can move buckets from one indexer to another. As long as you move the whole bucket, and only move warm or cold buckets. (I would suggest moving only cold buckets)

While strictly not necessary, I would personally do this with the indexers down. Also be very careful to not duplicate a bucket ID number.

As Kristian suggests, there are 5.0 features like accelerated search and index replication that could make this much messier. If you are using 5.0 features, I would suggest a run-through with support first.

Lucas_K
Motivator

It all really depends on your historical search usage so you need to think carefully about how to distribute your existing buckets. If not you could run into the situation where all your historical searches are being requested from a small number of search peers.
For moving from 9 to 15 indexes i'd probably just move 1 in 3 buckets to the other indexers. Make sure that you spread them out based on the bucket epoch times. This way a longer range search should hit more of your indexes at a time and thus increase search performance.

chris
Motivator

I remember being in a similar situation. I was worried that the existing indexers would fill up bevor everything would balance out. After having a closer look at the existing data we were able to delete quite a bit of things we didn't really need be adjusting the retention time for some indexes ... if that does not work for you, you could move a couple of indexes and then was as martin suggests.

martin_mueller
SplunkTrust
SplunkTrust

Moving single buckets around should work, but I wouldn't do it. Over time space will balance itself out, so I'd avoid the manual mess for the temporary balanced-ness.

kristian_kolb
Ultra Champion

I don't think that a bucket itself has knowledge of which index it belongs to, or on which server it resides. So in theory you could probably move them around to another indexer. However, this would/could mess with some stuff like accelerated searches, metadata, Deployment Monitor etc... so I'd check with support first. Or wait for a better answer here.

/k

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...