Getting Data In

Decomissioning indexers

brent_weaver
Builder

I am working to decommission some indexers from my cluster. I am using splunk offline --enforce-counts and letting them decommission on their own. I tried to do 8 at a time and it would not let me, but I could do one or two at a time and the others would just not decommission.

Any thoughts? Do I need to put the CM in maintainance mode?

0 Karma

ivanreis
Builder

It takes sometime until the indexer peer is completely stopped.

Here is how the splunk offline command works:

splunk offline: This is the fast version version of the splunk offline command. The peer goes down quickly, after a maximum of five minutes, even if searches or remedial activities are still in progress.
splunk offline --enforce-counts: This is the enforce-counts version of the command, which is designed to validate that the cluster has returned to the complete state. If you invoke the enforce-counts flag, the peer does not go down until all remedial activities are complete.

Further information -> https://docs.splunk.com/Documentation/Splunk/8.0.1/Indexer/Whathappenswhenaslavenodegoesdown

My suggestions is to remove the indexers from the cluster individually. Set the CM in maintenance mode and run "splunk offline" on the indexer you to remove.

Set CM in maintenance mode means that buckets hot buckets will not run on issues to roll up the bucket, so it is safe and recommended to setup this feature when you are doing any type of maintenance on the indexers.
More information about maintenance mode check this document -> https://docs.splunk.com/Documentation/Splunk/8.0.1/Indexer/Usemaintenancemode

Akeydel
Explorer

If you are decommissioning indexers, they have to redistribute all the data on them to other peers in the cluster. If you try to take down several all at once, that process will likely break.

Definitely decom them individually.

Also, if you're using  ./splunk offline --enforce-counts DO NOT set maintenance mode

  • The cluster cannot be in maintenance mode, because bucket fixup does not occur during maintenance mode.
https://docs.splunk.com/Documentation/Splunk/9.3.0/Indexer/Takeapeeroffline#The_enforce-counts_offli...
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...

Introduction to Splunk AI

How are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. Lucky for ...