We are setting up Splunk Cluster and wanted to know if Splunk recommends to use Autoscaling to launch N number of peer nodes and maintain required number of nodes in case of node failure also. Thanks in advance.
The trouble with Autoscaling Splunk is that you can only scale "up" and never "down" - once you bring a new indexer online and start using it, it will have data; turning it off means that you will lose data or at least force the cluster into a recovery state.
Remember that each indexer must have its own storage. You cannot merge the storage from two different indexers.
So even if you are only spinning up extra servers when you have experienced a failure, the new indexers that you spin up will have to stay in the cluster forever.
The way that the cluster makes the data highly available and reliable is by making extra copies. You want the cluster to be making the extra copies while it is up and running, and avoid rebuilding on the fly as much as possible.
So I don't think this is a very good idea in most cases. I am sure there is a way to make it work, and there might even be a compelling reason to do it - but I'm not seeing a good reason here...