Getting Data In

How to integrate AWS Autoscale with Splunk indexers to automate high availability without an admin redeploying configurations

Path Finder

Last year we had great luck with our Splunk configuration and I'm trying to adapt it to use multisite clustering for this year for a better HA story. There is one place where I'm getting stuck though.

There would be two indexers per AWS region in our setup. Ideally, these are set up to come up with an ASG in case one dies, they can automatically heal. I am not seeing a way for this to work without reconfiguring the forwarders with a new IP addresses when it comes up and it seems using an ELB in front of the indexers is frowned upon. Is there a known way to get this behavior so Splunk heals itself automatically without an admin going in and bringing up a new box and redeploying configurations?

0 Karma

Splunk Employee
Splunk Employee

Here is a .conf2015 talk that my colleagues and I did on deploying a highly available Splunk Enterprise architecture on AWS. We talk about how to leverage DNS entries instead of hardcoding IP addresses in your forwarders. Also, in Splunk 6.3 we introduced the new feature, indexer discovery, which allows the forwarders to get the full list of indexers from the master node.

Indexer Discovery Overview and Setup

Slidedeck from .conf2015 - Deploying Splunk on Amazon Web Services

Recording from .conf2015 - Deploying Splunk on Amazon Web Services

0 Karma


You're probably going to need to figure out some orchestration here. (And might already have some)

One thing that comes to mind is that you shouldn't be configuring your forwarders with IP addresses for each indexer. Instead, create a DNS listing with all of your indexer IP's as A records within it. Then you just point your forwarder at the DNS record, and it'll load-balance across all the IP's found.

When you need to add/remove indexers, you simply update the DNS listing. The forwarders will pick up on that change and forward to the new indexers automatically.

Relevant documentation

Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...