topic Re: Impossible to remove search head cluster in Deployment Architecture

Impossible to remove search head cluster

d123r432k — Thu, 24 Oct 2024 08:26:52 GMT

I configured a search head cluster and configured a captain and added the searchheads to the indexer cluster.

I now want to break the shcluster and have done this so far;

All from the cli:

removed the member that was not the captain, went ok

Tried to remove the other member, didnt work the command just hanged for half an hour before I gave up and aborted it.

Tried to set the captain in static mode, did a clean raft, but still no luck.

configured disabled=1 in the shclustering part of the server.conf and this time it went ok I guess

I now get the message this node is not a part of any cluster configuration.

Over to the indexer cluster where I now want to get rid of the searchheads from the GUI which is still showing up as up and running.

ran the command splunk remove cluster-search-heads and that went successful but the searchheads are still there in the indexer clustering GUI

some suggests that this will go away after a few minutes and after a restart of the manager node this will certainly go away. I have now waited a whole day and restarted, but they are still showing up and running with a green checkmark too.

Where does it get its information from and how can I get rid of them?

Re: Impossible to remove search head cluster

gcusello — Thu, 24 Oct 2024 16:31:32 GMT

Hi @d123r432k ,

you have to manually remove, from server.conf the SHC stanzas and restart the three SHs.

Ciao.

Giuseppe

Re: Impossible to remove search head cluster

d123r432k — Fri, 25 Oct 2024 05:23:27 GMT

edit the server.conf on the manager node or on the search heads?

Re: Impossible to remove search head cluster

d123r432k — Fri, 25 Oct 2024 12:54:39 GMT

I solved this by making a new searchhead cluster with the same machines with the same names. When I ran the command everything went fine

splunk edit cluster-config -mode searchhead -manager_uri https://10.152.31.202:8089 -secret newsecret123 -auth login:password

The problem was initially that I installed the deployer on the manager node. When I was about to install the enterprise security instance, it needed to be installed on the deployer for some reason. Now everything works as intended, I hope

Re: Impossible to remove search head cluster

gcusello — Fri, 25 Oct 2024 13:25:51 GMT

Hi @d123r432k ,

good for you, see next time!

let me know if I can help you more, or, please, accept one answer for the other people of Community.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉