Deployment Architecture

Setting up a Splunk indexer cluster, is it recommended to use Autoscaling?

sent2020
Explorer

We are setting up Splunk Cluster and wanted to know if Splunk recommends to use Autoscaling to launch N number of peer nodes and maintain required number of nodes in case of node failure also. Thanks in advance.

0 Karma

lguinn2
Legend

The trouble with Autoscaling Splunk is that you can only scale "up" and never "down" - once you bring a new indexer online and start using it, it will have data; turning it off means that you will lose data or at least force the cluster into a recovery state.

Remember that each indexer must have its own storage. You cannot merge the storage from two different indexers.

So even if you are only spinning up extra servers when you have experienced a failure, the new indexers that you spin up will have to stay in the cluster forever.

The way that the cluster makes the data highly available and reliable is by making extra copies. You want the cluster to be making the extra copies while it is up and running, and avoid rebuilding on the fly as much as possible.

So I don't think this is a very good idea in most cases. I am sure there is a way to make it work, and there might even be a compelling reason to do it - but I'm not seeing a good reason here...

Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...