topic Re: Load balancing REST API searches to a search head cluster in Splunk Dev

Can I load balancing REST API searches to a search head cluster?

mdwecht — Wed, 10 May 2023 19:53:15 GMT

Splunkers,

I have an external analytic engine that is currently making Splunk REST API calls to a specific search head in a search head cluster to pull data sets for analysis. It works great but I want to be able to load balance these REST calls across the search head cluster and each search requires a minimum of three REST calls to start the search, check the search status, and retrieve any available search results. I am sure I am not the first individual to require this functionality. Is this functionality already available in Splunk? Has anyone seen an open source implementation? Does a Phantom instance connect to a single Splunk search head? I don't want to degrade the user experience on a search head by having it dedicated to serving up data sets. Please advise...

Thanks,

Mark

Re: Load balancing REST API searches to a search head cluster

PickleRick — Mon, 06 Sep 2021 09:41:52 GMT

As per Splunk docs, for search-head cluster you should use a load-balancer that can keep a sticky session.

So if you intiate a first connection with no additional cookies added to it, you should get a session, or a server cookie (depending on how your LB is configured) and you should send this cookie with subsequent requests in order to get to the same backend (search-head).

Then for next search you again send initial request without the cookie, get cookie in response and sent it with additional REST calls.

Re: Load balancing REST API searches to a search head cluster

Akeydel — Wed, 10 May 2023 17:41:57 GMT

Do you have a link to those Splunk docs?

Re: Load balancing REST API searches to a search head cluster

PickleRick — Thu, 11 May 2023 06:56:17 GMT

https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/SHCarchitecture