Deployment Architecture

Search head cluster failure with 2 of 3 nodes - Can the user access Search head ?

splunker12er
Motivator

I have 3 nodes Search Head Cluster, User access the single FQDN and my F5 load balancer share the load to these 3 search heads .
If 2 out of 3 search heads nodes failed what would be the expected outcome ? (as per the docs its mentioned the Entire Cluster fails) - but my F5 will still share the load to the alive node ... in this case,

  1. Will the user can still able to access the alive search head node (1 alive) in my cluster ? and what would happen to the user search request ?

from the docs, link to splunk doc
When a member fails,
If a search head cluster member fails for any reason and leaves the cluster unexpectedly, the cluster can usually continue to function without interruption: The cluster's high availability features ensure that the cluster can continue to function as long as a majority (at least 51%) of the members are still running. For example, if you have a cluster configured with seven members, the cluster will function as long as four or more members remain up. If a majority of members fail, the cluster cannot successfully elect a new captain, which results in failure of the entire cluster. See "Search head cluster captain."

0 Karma

tiagofbmm
Influencer

Hey,

Maybe the trick here is the statement "functioning cluster". If you don't have a majority, then no dynamic captain will be elected, so without a captain elected you don't have any scheduled searches being dispatched by the captain (as this is his job) to the other members. In that sense, the cluster stops functioning.

Still, if you manage to elect that single member as a static captain ( or in the case of the docs, any of the remaining members as a static captain), then that one will still dispatch scheduled searches to himself and if you allow so, still do ad-hoc searches.

If you have one search left in your cluster, your users will still search the data.

Let me know if this helps

splunker12er
Motivator

Thanks . Yes, I am not interested in scheduled saved searches, I would need the users still can able to access the system and and do ad-hoc searches

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...