All Apps and Add-ons

bootstrapping process for splunk cluster on aws

ajaysamantbms
Explorer

I am referring to this article

https://github.com/splunk/splunk-aws-cloudformation

I am trying to understand the bootstrapping process of cluster

First bring up the master node
Then bring up search head ec2 instance

Before u spin up search head instance I would like
To update the configuration file on search head ec2
Instance to point to master node instance that just
Came up in previous step...I don't see that step
In this doc..

How would u typically do this step of updating conf
Files with ip addresses..in aws world?

Tags (2)
0 Karma
1 Solution

rarsan_splunk
Splunk Employee
Splunk Employee

Using Splunk AWS CloudFormation, the cluster configuration, as described in Splunk docs, is all automated by Chef using Splunk Cookbook. In a nutshell:

CloudFormation creates in AWS as many EC2 instances as needed each with assigned role (e.g. cluster master, search head, peer node, etc.) and provisions a new Chef server among other things. Each EC2 instance, as it bootstraps, checks in with the Chef server to receive instructions on how to configure itself, along with actual state of the system including ip addresses of other nodes such as the cluster master. Part of the instructions (i.e. Splunk Chef recipe), it will dynamically update the appropriate conf files with attributes such as ip addresses.

Hope this helps.

View solution in original post

0 Karma

rarsan_splunk
Splunk Employee
Splunk Employee

Using Splunk AWS CloudFormation, the cluster configuration, as described in Splunk docs, is all automated by Chef using Splunk Cookbook. In a nutshell:

CloudFormation creates in AWS as many EC2 instances as needed each with assigned role (e.g. cluster master, search head, peer node, etc.) and provisions a new Chef server among other things. Each EC2 instance, as it bootstraps, checks in with the Chef server to receive instructions on how to configure itself, along with actual state of the system including ip addresses of other nodes such as the cluster master. Part of the instructions (i.e. Splunk Chef recipe), it will dynamically update the appropriate conf files with attributes such as ip addresses.

Hope this helps.

0 Karma

ajaysamantbms
Explorer

I am going through the cookbook. Thanks.
Would you be able to point out the location in chef code where it gets the ip address from the master node splunk ec2 instance and uses that to configure the search head instance - just for example - i wanted to see how that logic works. how does the agent pass that info to recipe

I am trying to understand how the chef passes the ip address of the master node to search head and enters the ip address of the master node to create this section
[clustering]
master_uri = https://10.152.31.202:8089 in server.conf of search head- the ip address is dynamic.

0 Karma
Get Updates on the Splunk Community!

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...