Splunk Enterprise

remove excess buckets

jariw
Path Finder

Hi,

some questions...

Last weekend we've got an error on the indexers. It is a multisite indexers with 6<>6 indexers (each site 6 indexers). 

Some indexers went down and the data storage went sky high. Stil not sure why. But when we started the indexers which where down, the data storage went partly back on nomal except one indexer.

I noticed a lot off excess buckets...  very very much. I started removing these buckets,  but it stopped on one point and never went further with cleaning.  Could this be because of the data part of this one indexer is full (it is at this moment in automatic detention state).

I don't see the activity on the cluster master, so it seems it is finished.. but i can't start a new action to clean, is says "Previously scheduled Remove Excess Buckets is running". 

I tried a rolling restart (in maint mode), but it doesn't allow because of the "remove excess buckets is running"..

How can i stop this "Previously scheduled Remove Excess Buckets is running"  ?

thanks in advance..

Labels (2)
0 Karma

jariw
Path Finder

It solved itself after restart two ther indexers and remove excess buckets per index (what strangly was allowed).

After a while it seems there was enough space for removing the rest off the excess buckets.

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Usually you could fix this kind of situations by rebooting CM.
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...