topic Re: Splunk Cluster Master Server Fault Tolerance in Deployment Architecture

Splunk Cluster Master Server Fault Tolerance

micatcloudon — Sun, 30 Dec 2012 13:59:34 GMT

Greetings!
While the fault tolerance for the Splunk Cluster node is clear.
I wonder How Splunk Cluster would recover from the Splunk Master server loss ?
What is the remedy procedure ?

Re: Splunk Cluster Master Server Fault Tolerance

roychen — Mon, 31 Dec 2012 04:28:38 GMT

Hi,

You simply need to restore the master node, and the peers in the cluster will communicate with the master regarding the current state of the cluster (number of replicated copies / searchable copies, etc).

To make sure you can restore a master node ASAP, you might want to keep a standby master node, as per the instructions in:

http://docs.splunk.com/Documentation/Splunk/5.0.1/Indexer/Configurethemaster#Configure_a_stand-by_master

Hope this helps.

Re: Splunk Cluster Master Server Fault Tolerance

Drainy — Mon, 31 Dec 2012 10:00:09 GMT

Essentially you can start up a new master node with the same IP/DNS details and it will assume the role of the original as the other answer aludes to, however the truth is that the master itself has no HA or fault tolerance.

If you were to experience an event which took a master and an indexer offline you would be left in an inconsistant state, as per Starfleets situation here; http://splunk-base.splunk.com/answers/65397/splunk-v5-clustering-and-ha

Its still a much improved HA solution and its pretty good to be fair, but there really needs to be better tolerance for the master failing 🙂