Installation

Events might not be returned in sub-second order due to search memory limits

rayar
Contributor
we are facing the below issue for searches 
please advise what I can reconfigure to avoid this errors   ?

4 errors occurred while the search was executing. Therefore, search results might be incomplete. 
  • [ilissplidx01] Events might not be returned in sub-second order due to search memory limits. See search.log for more information. Increase the value of the following limits.conf setting:[search]:max_rawsize_perchunk.
  • [ilissplidx02] Events might not be returned in sub-second order due to search memory limits. See search.log for more information. Increase the value of the following limits.conf setting:[search]:max_rawsize_perchunk.
  • [ilissplidx06] Events might not be returned in sub-second order due to search memory limits. See search.log for more information. Increase the value of the following limits.conf setting:[search]:max_rawsize_perchunk.

[splunk@ilisspldepl01 deployment-apps]$ cat ./AM_all_indexers_tuning/local/limits.conf
[default]
max_mem_usage_mb = 600
#
[search]
#dispatch_dir_warning_size = 3500
base_max_searches = 60
# # ERROR: Events may not be returned in sub-second order due to memory pressure.
max_rawsize_perchunk = 200000000
#
[pdf]
max_rows_per_table = 10000
#
[scheduler]
max_searches_perc = 100
#
[join]
subsearch_maxout = 500000
#
[realtime]
indexed_realtime_use_by_default = true
[splunk@ilisspldepl01 deployment-apps]$



  • [ilissplidx08] Events might not be returned in sub-second order due to search memory limits. See search.log for more information. Increase the value of the following limits.conf setting:[search]:max_rawsize_perchunk.

    [splunk@ilissplidx01 ~]$ grep MemTotal /proc/meminfo
    MemTotal: 65688816 kB
    [splunk@ilissplidx01 ~]$
    [rayar@ilissplidx08 ~]$ grep MemTotal /proc/meminfo
    MemTotal: 528052452 kB
    [rayar@ilissplidx08 ~]$
0 Karma

scelikok
SplunkTrust
SplunkTrust

It depends on your events size and number of events per seconds.  You can try by 200000000 increments.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @rayar,

Actually this is a warning that you probably face when you run a search that gets all raw data. If you run a search that has statistics command like stats, you would not see that error. 

I suggest to check event ingestion if timestamps are correctly parsed. This may due to large numbers of events with the exact same timestamp -- possibly caused by non-timestamped events that are being timestamp by Splunk as they are indexed.

If timestamps are ok, you want to retrieve all raw data on search you can try increasing "max_rawsize_perchunk" to much higher value. 

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

rayar
Contributor

this index contains the data with current_time sourcetype 

the question what value you would recommend to set for max_rawsize_perchunk 

0 Karma

microgag
Engager
0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...