How to increase the maximum number of real-time searches

[Bliide]

I am trying to do a stress test on a new server in a fresh Splunk environment. I would like to increase the number of real-time searches allowed and see how much of a performance hit the server takes. I know I need to edit limits.conf, but I am not certain what stanza to add and what values to use.

Anyone with experience in tweaking limits.conf is what I am looking for. Is it best to start off by just adding a search stanza with: max_rt_search_multiplier = 2

or is it better to add:

max_searches_per_cpu = 2 ?

Our goal to is see how many concurrent real-time searches we can run before we start having a substantial performance hit.

[steveyz]

changing the max_rt_search_multiplier is the way to go. changing max_searches_per_cpu and base_max_searches will change also affect the real-time limit, but will alter the limit for historical searches too.

max real-time searches = max_rt_search_multiplier x (max_searches_per_cpu * + base_max_searches)

And to address woodcock's comment, there is a setting that you can tweak which will allow you to optionally trade performance for latency. It will run real-time searches with higher latency but generally use far less system resources. The setting is under

[realtime]
indexed_realtime_use_by_default = true/false (defaults to false. set to true for less resource usage but higher latency)

[woodcock]

I can tell you that answer: ONE! Unless you have designated your entire cluster to the purpose of running Real-Time Searches, don't run any.