Splunk Enterprise

remove excess buckets

jariw
Path Finder

Hi,

some questions...

Last weekend we've got an error on the indexers. It is a multisite indexers with 6<>6 indexers (each site 6 indexers). 

Some indexers went down and the data storage went sky high. Stil not sure why. But when we started the indexers which where down, the data storage went partly back on nomal except one indexer.

I noticed a lot off excess buckets...  very very much. I started removing these buckets,  but it stopped on one point and never went further with cleaning.  Could this be because of the data part of this one indexer is full (it is at this moment in automatic detention state).

I don't see the activity on the cluster master, so it seems it is finished.. but i can't start a new action to clean, is says "Previously scheduled Remove Excess Buckets is running". 

I tried a rolling restart (in maint mode), but it doesn't allow because of the "remove excess buckets is running"..

How can i stop this "Previously scheduled Remove Excess Buckets is running"  ?

thanks in advance..

Labels (2)
0 Karma

jariw
Path Finder

It solved itself after restart two ther indexers and remove excess buckets per index (what strangly was allowed).

After a while it seems there was enough space for removing the rest off the excess buckets.

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Usually you could fix this kind of situations by rebooting CM.
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...