Deployment Architecture

How to configure Load Balancing on Splunk Search Heads?

agentsofshield
Path Finder

Hi! So I set up a F5 Load Balancer and listed all of my Splunk search heads as pool members.

Apparently the load balancer performs a health check, and therefore, requires a health monitor URI and a health monitor response!

So I'm consulting you guys, which URI and response should I use? It's just a simple request and response to check if my search head is up. With the default configurations my server is considered down, of course.

I have no experience with load balancers so please be gentle.

jkat54
SplunkTrust
SplunkTrust

I recommend monitoring the Splunkd port of 8089 (tls), because when it isnt responding it might still accept requests on the web port, but not be able to service them.

0 Karma

codebuilder
Motivator

This is older but looks like it never got a solid answer. The easiest solution is to perform a simple port check.

I don't use F5, but the haproxy equivalent is such:

balance source
server shca101 :8000 weight 1 maxconn 2000 check port 8000 ssl verify none
server shca102 :8000 weight 1 maxconn 2000 check port 8000 ssl verify none
server shca103 :8000 weight 1 maxconn 2000 check port 8000 ssl verify none

Before sending traffic to any of the search heads the load balancer verifies that port 8000 is active. Assuming your web UI is using the default port 8000 of course.

0 Karma

joesrepsolc
Communicator

Struggling with this same scenario... and still have no solid answer. Anyone have more info?

Thanks.

Joe

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Hi @agentsofshield ,

The simplest method is to configure an http monitor to check the service.

Select http as the health monitor and set GET /\r\n as your send string in properties. This is very basic check but should work

agentsofshield
Path Finder

I don't have a send string, I can only fill out a URI and a response.

The team in charge of servers / load balancers has a system so I have little control over it.

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Ok. In URI you could mention / which is the root context and in response you can leave blank.

0 Karma

Tune In & Win!

Don't miss out on your
chance to take home free
prizes by helping our players
save the Splunk Cloudom!

Dungeons & Data
Monsters: Splunk O11y
Day Editions Games
stream live:
5/4 at 6:30pm PST
5/5 at 7:00pm PST
on