I recently started pulling Tenable data in through the Tenable Add-on for Splunk, and when I search the data in Search and Reporting it produces this error.
"[FQDN.domain.mil,FQDN.domain.mil] Events might not be returned in sub-second order due to search memory limits.
See search.log for more information. Increase the value of the following limits.conf setting:[search]:max_rawsize_perchunk."
I searched through search.log and found no further information, and I am able to search Tenable data. What I want to know is how this ultimately effects search performance or data searchability/availability? Can I clear this error by simply increasing the max_rawsize_perchunk in the props.conf?
@Jtorge This error occurs because Splunk is under memory pressure when handling large events (scan outputs from tenable), so it may not return results in perfect sub‑second order. Your data is still indexed and searchable, hence availability is not affected. The impact is mainly on search performance and event ordering.
To address it, you can try the below: [Do it in Dev instances first and then to prod when applying settings]
First, review the data being collected and look for opportunities to reduce event size before ingestion (e.g., adjust Tenable export chunking) or use indexed extractions or field filtering to avoid massive raw events from scan results which is not meeting any use case or detection for you.
If the error still persists post reduction efforts, adjust the max_rawsize_perchunk setting in limits.conf. The default is 100 MB. if your events are larger, increase it conservatively (e.g., 200 MB) and monitor system performance. A common rule of thumb is to keep the value at no more than 10% of your index bucket size.
Please note that raising the limit clears the warning but can increase memory usage, so balance the setting against available resources.
The below reference would be helpful.
Ref: Events might not be returned in sub-second order due to search memory limits. | Splunk
>>
If this post addressed your question, you can:
Acknowledging helpful answers keeps the community strong and motivates contributors to continue sharing their expertise.
>>
@Jtorge This error occurs because Splunk is under memory pressure when handling large events (scan outputs from tenable), so it may not return results in perfect sub‑second order. Your data is still indexed and searchable, hence availability is not affected. The impact is mainly on search performance and event ordering.
To address it, you can try the below: [Do it in Dev instances first and then to prod when applying settings]
First, review the data being collected and look for opportunities to reduce event size before ingestion (e.g., adjust Tenable export chunking) or use indexed extractions or field filtering to avoid massive raw events from scan results which is not meeting any use case or detection for you.
If the error still persists post reduction efforts, adjust the max_rawsize_perchunk setting in limits.conf. The default is 100 MB. if your events are larger, increase it conservatively (e.g., 200 MB) and monitor system performance. A common rule of thumb is to keep the value at no more than 10% of your index bucket size.
Please note that raising the limit clears the warning but can increase memory usage, so balance the setting against available resources.
The below reference would be helpful.
Ref: Events might not be returned in sub-second order due to search memory limits. | Splunk
>>
If this post addressed your question, you can:
Acknowledging helpful answers keeps the community strong and motivates contributors to continue sharing their expertise.
>>