Splunk Search

Why am I getting "message Max Raw Size Limit Exceeded" errors that are now affecting search performance?

gwobben
Communicator

Hi Guys,

I'm running a search and it seems to take longer than needed. I've search the logs for errors and found this in search.log:

ERROR IndexReaderIf - Max Raw Size Limit Exceeded
INFO  UnifiedSearch - Error in 'IndexReaderIf': Max Raw Size Limit Exceeded

It's generating 100s of these messages in pairs, but I don't understand what it means. What can I do to solve this error?

Details:
Single node Splunk install
Splunk 6.3.0
8 CPU cores
32 GB RAM

davidpaper
Contributor

Greetings,

Cribbing this from some Support folks who have run into this before. There is a setting that can be updated to alleviate the message.

$SPLUNK_HOME/etc/system/local/limits.conf (or wherever you maintain infrastructure changes like this, but definitely not in etc/system/default/limits.conf!)

[search]
max_rawsize_perchunk =

You need to restart Splunk for your change to take effect.

If you have enough memory available that you feel comfortable allowing searches to use more of it, you can set 0 as unlimited. We don't have any recommendation threshold about max_rawsize_perchunk, as it depends on how much memory is available on the hosts for the processes to consume. Default is 100MB. If you don't feel comfortable with unlimited, double it until the error message disappears. It's a little bit of trial and error, yes.

jrodman
Splunk Employee
Splunk Employee

It sure would be great if someone could communicate what behavior is known to correlate with this message. That is, is it a nuisance? Is this a performance affecting situation? do you get incomplete search results?

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...