I am automating removing a search head (SH) instance from both the search head cluster and the index cluster. Removing the SH from the shcluster is easy, but removing the SH from the index cluster seems problematic -- assuming it is done via a CLI command and not by restarting the cluster master instance.
The SH was added to the index cluster using the following command from the SH:
./splunk edit cluster-config -auth admin:${PASSWD} -mode searchhead -site ${SITE} -master_uri ${CLUSTERMSTR_URI} -replication_port 8080 -secret ${CLUSTERMSTR_PASSWD}
Also from the SH, am trying to remove the SH from the index cluster using:
./splunk remove cluster-master -master_uri https://splunk-clustermstr-dev.fmrco.com:8089 -secret ${CLUSTER_PWD}
result: In handler 'clustersearchheadconfig': Cannot remove this master.
I also tried to run the following command from the cluster master:
./splunk remove cluster-peers -peers 8C842081-A220-4081-87AC-92915D567845
Result:
In handler 'clustermastercontrol': Remove aborted, Reason: Cannot find peer with guid=8C842081-A220-4081-87AC-92915D567845 to remove.
Wasn't sure why till running ./splunk list cluster-peers and saw that it is only showing indexer entries, not SH entries.
I am hoping not to have to restart the cluster master instance to get rid of removed SH instances -- let me know how please.
Regards.
... View more