Deployment Architecture

Search head cluster failure with 2 of 3 nodes - Can the user access Search head ?


I have 3 nodes Search Head Cluster, User access the single FQDN and my F5 load balancer share the load to these 3 search heads .
If 2 out of 3 search heads nodes failed what would be the expected outcome ? (as per the docs its mentioned the Entire Cluster fails) - but my F5 will still share the load to the alive node ... in this case,

  1. Will the user can still able to access the alive search head node (1 alive) in my cluster ? and what would happen to the user search request ?

from the docs, link to splunk doc
When a member fails,
If a search head cluster member fails for any reason and leaves the cluster unexpectedly, the cluster can usually continue to function without interruption: The cluster's high availability features ensure that the cluster can continue to function as long as a majority (at least 51%) of the members are still running. For example, if you have a cluster configured with seven members, the cluster will function as long as four or more members remain up. If a majority of members fail, the cluster cannot successfully elect a new captain, which results in failure of the entire cluster. See "Search head cluster captain."

0 Karma



Maybe the trick here is the statement "functioning cluster". If you don't have a majority, then no dynamic captain will be elected, so without a captain elected you don't have any scheduled searches being dispatched by the captain (as this is his job) to the other members. In that sense, the cluster stops functioning.

Still, if you manage to elect that single member as a static captain ( or in the case of the docs, any of the remaining members as a static captain), then that one will still dispatch scheduled searches to himself and if you allow so, still do ad-hoc searches.

If you have one search left in your cluster, your users will still search the data.

Let me know if this helps


Thanks . Yes, I am not interested in scheduled saved searches, I would need the users still can able to access the system and and do ad-hoc searches

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...