Good evening all.
I would like to know exactly how to properly tear down a search head cluster.
I am rebuilding / upgrading a Splunk environment I inherited and I need to re-utilize some of the servers in the seach head cluster for dedicated purposes; such as installed Enterprise security app on a dedicated search head.
I have thought this through long a hard and decided this is the direction I want to go.
I have already disabled search head cluster on the members (set disabled stanza to 1 in the shclustering stanzas.)
As soon as I did this, I am now getting KV store failed errors on all 3 search heads.
I did the KVstore status command and the status show failed on the status details.
I am not sure exactly where to go from here but I am sure that disabling search head clustering on my search heads caused the KV store issue.
How do you "undo/bring down" a search head cluster completely; or maybe its better to say "revert" the search head cluster back into individual search heads?
Could the KVStore errors really be associated with my disabling the search head members?
If not, where could this KV error be coming from?
Thanks for the response.
Turn the "disabled" setting back to 0 for all search heads; logged into one to verify and saw the "search head clustering" option available again in the UI.
Then tried to remove as you stated; no joy, still getting the KV errors.
It seems like once you get it into search head clustering, its hard to completely break it down so its as though you never configured it at all; which is what I am aiming to do.
Do I have to dis-engage the conf_deploy_fetch_uri (cluster deployer) too or any other conf file modifications on that server?