Deployment Architecture

What is recommended for migrating from my current Search Head Pooling setup to Search Head Clustering?

mfrost8
Builder

As of Splunk 6.2, I see that search head pooling has been deprecated so I need to consider changing course from the infrastructure we've already embarked upon and think about how do I go from pooling to clustering for search heads.

Right now the SH pooling I'm doing is pretty light but poised to be expanded. The most we have is 2 pooled servers behind a load balancer. I see there's different Splunk configuration (expected) and it appears that the requirement for NFS storage goes away.

In terms of thinking of a transition period, it looks like the minimum for SH clustering is 3 cluster members. So if I've got only 2 servers (and need only 2 at present), I take it this means I have to build a 3rd SH to be able to move to SH clustering? Or did I misunderstand the documentation I read?

Also, it's not practical to mount NFS volumes across a WAN so we've created unique pools per geographic datacenter. Does SH clustering now make it practical to create a cross-WAN cluster so that no matter which geographic search head a user logs in to they will get their same saved searches, dashboards, etc?

Thanks

1 Solution

awilliams_splun
Splunk Employee
Splunk Employee

Yes, a minimum of 3 systems is required for SH Clustering. This is largely do to the SH Cluster captain election process. In SH cluster the SH cluster has what is called a Captain. The captain is used to coordinate activities across the SH Cluster to include:

  • Schedule jobs across the cluster
  • Push knowledge bundles to peers
  • Replicate the various search artifacts to peers
  • Replicate any configuration changes made to peers (the use of deployer server is required for config changes. system side config changes should not be made locally)
  • Coordinating alerts
  • etc

When the system is brought up for the first time or the scheduled/unscheduled restart of the captain, a election process occurs to elect a new captain. This election process requires a 51% majority vote from all members in the cluster. This would not be achievable in a two node cluster.

View solution in original post

jnicholsenernoc
Path Finder

Are people successfully deploying search head clustering over the WAN?

0 Karma

ppablo
Retired

Hi @jnicholsenernoc

Please don't use the answer space on someone else's post to ask a question. Please post your question as a completely new post.

0 Karma

awilliams_splun
Splunk Employee
Splunk Employee

Yes, a minimum of 3 systems is required for SH Clustering. This is largely do to the SH Cluster captain election process. In SH cluster the SH cluster has what is called a Captain. The captain is used to coordinate activities across the SH Cluster to include:

  • Schedule jobs across the cluster
  • Push knowledge bundles to peers
  • Replicate the various search artifacts to peers
  • Replicate any configuration changes made to peers (the use of deployer server is required for config changes. system side config changes should not be made locally)
  • Coordinating alerts
  • etc

When the system is brought up for the first time or the scheduled/unscheduled restart of the captain, a election process occurs to elect a new captain. This election process requires a 51% majority vote from all members in the cluster. This would not be achievable in a two node cluster.

MartinMcNutt
Communicator

Also note....Windows is not a support Operating system for SH Cluster as of 6.2.1.

0 Karma

phoffman_splunk
Splunk Employee
Splunk Employee
0 Karma

mfrost8
Builder

So ultimately, once search head pooling is gone, you can run 1 standalone search head, or 3+ search heads in a clustered configuration. There is no solution whereby you can use 2 search heads (even if you don't need 3).

Thanks.

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...