topic Re: Master node failure?? in Deployment Architecture

Master node failure??

AzmathShaik — Thu, 14 Jul 2016 14:42:33 GMT

Hello
we are planning to move to clusters so i have gone through document about the setup. i have a question if Master Node is failed is there any way that master node is replace as soon as the original one goes down (that configuring the stand by master node in passive state and bring it up as soon as original one goes down).

Re: Master node failure??

Steve_G_ — Thu, 14 Jul 2016 15:28:15 GMT

This topic should answer your question:

http://docs.splunk.com/Documentation/Splunk/6.4.2/Indexer/Handlemasternodefailure

Re: Master node failure??

AzmathShaik — Thu, 14 Jul 2016 15:32:56 GMT

To bring the master up do we need to perform the actions manually??

Re: Master node failure??

Steve_G_ — Thu, 14 Jul 2016 15:40:44 GMT

There's probably some tool you can use to automate at least part of the process, but it's a fairly short series of steps that you need to execute.

One thing to realize... The cluster can usually tolerate a failed master for a period of time before significant problems arise. So, you do not need a new master to pop up immediately if the current one fails. But, as the previously cited topic describes, you do need to prepare ahead of time for the possibility of maser failure.

For details on how the cluster reacts to a master failure, see:

http://docs.splunk.com/Documentation/Splunk/6.4.2/Indexer/Whathappenswhenamasternodegoesdown

Re: Master node failure??

AzmathShaik — Thu, 14 Jul 2016 16:25:17 GMT

Thanks for your answer

one last question may i know what is the default time limit that a master node can tolerate and does that parameter can be changed??? from default value

Re: Master node failure??

Steve_G_ — Thu, 14 Jul 2016 16:58:11 GMT

I'm not sure what you mean by a "default time limit." How long a cluster can tolerate a downed master depends on what else is happening with the cluster, as described here:

http://docs.splunk.com/Documentation/Splunk/6.4.2/Indexer/Whathappenswhenamasternodegoesdown

Re: Master node failure??

sbrice17 — Wed, 26 Jun 2019 14:53:15 GMT

What if you introduced a GTM and LB in the model?

GTM talks to the two LB's, one on each side. One VIP is used for both CM's.

Primary route is to CM-1 (site 1 master)

LB monitors the health status of CM-1(site1) , when it see's a failure on the CM-1 it then routes the traffic to the CM2(Site2)

This model would make the CM-2 (site2) a Hot standby, would the indexers still require a rolling restart, or could things keep on flowing as normal?