All Apps and Add-ons

bootstrapping process for splunk cluster on aws

ajaysamantbms
Explorer

I am referring to this article

https://github.com/splunk/splunk-aws-cloudformation

I am trying to understand the bootstrapping process of cluster

First bring up the master node
Then bring up search head ec2 instance

Before u spin up search head instance I would like
To update the configuration file on search head ec2
Instance to point to master node instance that just
Came up in previous step...I don't see that step
In this doc..

How would u typically do this step of updating conf
Files with ip addresses..in aws world?

Tags (2)
0 Karma
1 Solution

rarsan_splunk
Splunk Employee
Splunk Employee

Using Splunk AWS CloudFormation, the cluster configuration, as described in Splunk docs, is all automated by Chef using Splunk Cookbook. In a nutshell:

CloudFormation creates in AWS as many EC2 instances as needed each with assigned role (e.g. cluster master, search head, peer node, etc.) and provisions a new Chef server among other things. Each EC2 instance, as it bootstraps, checks in with the Chef server to receive instructions on how to configure itself, along with actual state of the system including ip addresses of other nodes such as the cluster master. Part of the instructions (i.e. Splunk Chef recipe), it will dynamically update the appropriate conf files with attributes such as ip addresses.

Hope this helps.

View solution in original post

0 Karma

rarsan_splunk
Splunk Employee
Splunk Employee

Using Splunk AWS CloudFormation, the cluster configuration, as described in Splunk docs, is all automated by Chef using Splunk Cookbook. In a nutshell:

CloudFormation creates in AWS as many EC2 instances as needed each with assigned role (e.g. cluster master, search head, peer node, etc.) and provisions a new Chef server among other things. Each EC2 instance, as it bootstraps, checks in with the Chef server to receive instructions on how to configure itself, along with actual state of the system including ip addresses of other nodes such as the cluster master. Part of the instructions (i.e. Splunk Chef recipe), it will dynamically update the appropriate conf files with attributes such as ip addresses.

Hope this helps.

0 Karma

ajaysamantbms
Explorer

I am going through the cookbook. Thanks.
Would you be able to point out the location in chef code where it gets the ip address from the master node splunk ec2 instance and uses that to configure the search head instance - just for example - i wanted to see how that logic works. how does the agent pass that info to recipe

I am trying to understand how the chef passes the ip address of the master node to search head and enters the ip address of the master node to create this section
[clustering]
master_uri = https://10.152.31.202:8089 in server.conf of search head- the ip address is dynamic.

0 Karma
Get Updates on the Splunk Community!

New Splunk Observability innovations: Deeper visibility and smarter alerting to ...

You asked, we delivered. Splunk Observability Cloud has several new innovations giving you deeper visibility ...

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...