Deployment Architecture

Tearing down a search head cluster

wegscd
Contributor

I am going to have to tear down our SHC; I had to give up our 3rd search head to another project, which puts us in un-supported land (2 node cluster). I plan to split the deployed apps between the 2 remaining search heads.

Right now, I plan is to back up $SPLUNK_HOME/etc/apps on each of the two remaining nodes, remove the [shcluster] stanzas, then go through all the apps, comparing the contents of /etc/apps//local and /etc/apps//default on the search heads with what was getting pushed from the deployer, making sure that the default on the search heads matches the default from the deployers, and the changes in local on the deployers are merged into the local changes on the search heads.

  • Is this realistic?
  • is anyone else aware of any gotchas here?
  • what do I need to do on the deployer to tear down the shcluster?
0 Karma

jmallorquin
Builder

Hi,

Have you valorate the option of make 2 more instances of search head in the two servers, to have 4 search head in 2 machines?

Just to keep the search head cluster?

Hope help you.

0 Karma

jmallorquin
Builder

Hi,

As you know, all the files in the deployer from default and local are merge to the default folder in ther search head. So you only need to check the default and local in ther search head.

Hope help you.

0 Karma

wegscd
Contributor

I am trying to get the applications back to a state of original install. If I tear the cluster down and leave local changes (from the deployer) in default.conf, I will lose them in the next upgrade....

0 Karma

jmallorquin
Builder

Hi,

If you have a local configuration in the search head this will no apply

"making sure that the default on the search heads matches the default from the deployers" couse the changes in the shc are not pushing to the deployer.

0 Karma

wegscd
Contributor

why won't this apply? won't I want both the local changes from the deployer, and any local changes that have been made during the operation of the cluster?

0 Karma

wegscd
Contributor

Had not considered that; not sure that I want to deal with the complexity...

0 Karma
Get Updates on the Splunk Community!

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...

September Community Champions: A Shoutout to Our Contributors!

As we close the books on another fantastic month, we want to take a moment to celebrate the people who are the ...

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...