Splunk Search

How to increase the maximum real-time concurrent searches limit?

f_luciani
Path Finder

Hi,

I've been using Splunk 6.1.2 trial for a week now, it has been installed on Debian and is running fine, but... I've started getting the following message:

The maximum number of real-time
concurrent system-wide searches has
been reached. current=10 maximum=8

...which, in turn, prevents me from creating more dashboards, since every new search I create goes to a queue, not running unless I get rid of the other, real-time searches.

I've been investigating and found out about max_searches_per_cpu, base_max_searches and max_rt_search_multiplier options of limits.conf, tried to change it to no avail (oddly enough, my $SPLUNK_HOME/etc/system/local/limits.conf was empty, copied the options from the one within /default directory and changed the default values to allow more searches). The messages I get when running './splunk btool check --debug' are:

...
    Checking: /opt/splunk/etc/system/local/limits.conf
        Invalid key in stanza [inputproc] in /opt/splunk/etc/system/loca  l/limits.conf, line 5: max_searches_per_cpu  (value:  2)
        Did you mean 'max_fd'?
        Did you mean 'max_mem_usage_mb'?
        Did you mean 'min_batch_size_bytes'?
    Invalid key in stanza [inputproc] in /opt/splunk/etc/system/loca  l/limits.conf, line 8: base_max_searches  (value:  12)
    Invalid key in stanza [inputproc] in /opt/splunk/etc/system/loca  l/limits.conf, line 11: max_rt_search_multiplier  (value:  2)
        Did you mean 'max_fd'?
        Did you mean 'max_mem_usage_mb'?
        Did you mean 'min_batch_size_bytes'?
...

I've also tried to change the values for 'power' role (top right corner, Settings -> Users and authentication -> Access controls -> Roles -> Power -> 'User-level concurrent real-time search jobs limit' & 'Role-level concurrent real-time search jobs limit' options), doubled the values but nothing happened, still the same message. This is happening because I have 5 dashboards with real-time searches opened in my screen and the same 5 in a big flat screen for the IT personnel, making it 10 searches in total (even thought I proceeded login in both web interfaces as admin user, it seems Splunk adds up all searches, regardless of the fact the web interface sessions in different machines belong to the same user, which I regard as normal behaviour, actually).

Has anyone encountered this issue as well? What am I doing wrong here? I would like to increase max concurrent rt searches, any help would be appreciated.

Thanks in advance.

1 Solution

yannK
Splunk Employee
Splunk Employee

I suppose that you are missing the stanza name before the parameter. In your case this is the stanza "search"

see .../local/limits.conf

[search]
max_searches_per_cpu =2

View solution in original post

yannK
Splunk Employee
Splunk Employee

I suppose that you are missing the stanza name before the parameter. In your case this is the stanza "search"

see .../local/limits.conf

[search]
max_searches_per_cpu =2

f_luciani
Path Finder

It worked like a charm, thanks.

0 Karma

f_luciani
Path Finder

I suppose you are right... I'll give it a go on Monday just to be sure and report back here to let you know how it ended. Thanks a lot for your help so far.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

🍂 Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...