topic Re: How do you calculate max search concurrency in a search head cluster and an indexer cluster environment? in Deployment Architecture

How do you calculate max search concurrency in a search head cluster and an indexer cluster environment?

yutaka1005 — Fri, 01 Feb 2019 02:32:08 GMT

I know that how to calculate max search concurrency in stand-alone is below.

normal search : max_hist_searches = max_search_per_cpu(* default is 1) * core + base_max_searches(* default is 6)

normal real-time search : max_realtime_searches = max_rt_search_multiplier(* default is 1) * max_hist_searches

saved search : max_hist_scheduled_searches = max_searches_perc(* default is 50)/100 * max_hist_searches

saved real-time search : max_realtime_scheduled_searches = max_searches_perc(* default is 50)/100 * max_realtime_searches

But if there is an environment such as below, how would I calculate?

Search head : 3 (* contains captain)
Indexer : 4 (* not contains Cluster master)

Please someone tell me.

Also if there is document that mentions about it, please tell me too.

Re: How do you calculate max search concurrency in a search head cluster and an indexer cluster environment?

MuS — Fri, 01 Feb 2019 05:39:03 GMT

Not quiet sure what you are asking, but the rules are exactly the same ¯\_(ツ)_/¯
Maybe summoning @YamadaNoel can help with the language here ?

cheers, MuS

Re: How do you calculate max search concurrency in a search head cluster and an indexer cluster environment?

vishaltaneja070 — Fri, 01 Feb 2019 06:15:07 GMT

Hello @yutaka1005,

The below formula works for all:

 max_searches_per_cpu x number_of_cpus + base_max_searches = max_hist_searches
 (max_searches_perc / 100) x max_hist_searches = max_hist_scheduled_searches
 max_rt_search_multiplier x max_hist_searches = max_realtime_searches

The default settings are base_max_searches = 6, max_searches_per_cpu = 1, max_searches_per = 50, and max_rt_search_multiplier = 1. These settings can be changes using limits.conf but you need to be careful as it all depends on your hardware capability:

 ((1 * 8) +6) = 14 = max_hist_searches per SH
 3 * 14 = 42 = max search head cluster concurrency

((50 /100) * 14 ) = 7 = max_historical_scheduled_searches per SH
 3 * 7 = 21 max search head cluster scheduled search concurrency

 1 * 14 = 14 = max_realtime_searches per SH
 3 * 14 = 42= max rt search head cluster concurrency

This Link can provide you better understanding:

https://docs.splunk.com/Documentation/Splunk/7.2.3/DistSearch/SHCarchitecture

Also, if captain feels the SHC is not able to handle that much jobs, it will start delaying or cancel the jobs automatically.

Re: How do you calculate max search concurrency in a search head cluster and an indexer cluster environment?

rohitvjoshi — Fri, 01 Feb 2019 06:45:29 GMT

Hi ,

Concurrency depends on no of cpu cores /indexers we have,below the table for understanding:
IDX CPU Cores Concurrency
IDX-1 5 5
IDX-2 10 10

When we run a serach it takes 1 core to show the results.so we can say concurrency directely proposnal to number of cpu cores .

Thanks

Re: How do you calculate max search concurrency in a search head cluster and an indexer cluster environment?

yutaka1005 — Mon, 04 Feb 2019 05:47:47 GMT

Thank you for answer.

I understood that same rules as stand-alone are applied to each search heads, and the captain load balances searches that is handled by each search head.

It is very helpful.

Re: How do you calculate max search concurrency in a search head cluster and an indexer cluster environment?

yutaka1005 — Mon, 04 Feb 2019 05:52:06 GMT

Sorry, I'm not good at english.

I wanted to know whether the same rule(*max concurrency of parallel search) as the stand-alone applies to the environment where the SH cluster and the indexer cluster exist.

So, problem is solved.

Re: How do you calculate max search concurrency in a search head cluster and an indexer cluster environment?

vishaltaneja070 — Mon, 04 Feb 2019 06:28:20 GMT

@yutaka1005

Correct same rules as of stand-alone environment.

Thanks 🙂