Getting Data In

Events not returned in sub-second order

Jtorge
Explorer

I recently started pulling Tenable data in through the Tenable Add-on for Splunk, and when I search the data in Search and Reporting it produces this error.

"[FQDN.domain.mil,FQDN.domain.mil] Events might not be returned in sub-second order due to search memory limits.
See search.log for more information. Increase the value of the following limits.conf setting:[search]:max_rawsize_perchunk."

I searched through search.log and found no further information, and I am able to search Tenable data. What I want to know is how this ultimately effects search performance or data searchability/availability? Can I clear this error by simply increasing the max_rawsize_perchunk in the props.conf?

Labels (2)
0 Karma
1 Solution

kknairr
Contributor

@Jtorge This error occurs because Splunk is under memory pressure when handling large events (scan outputs from tenable), so it may not return results in perfect sub‑second order. Your data is still indexed and searchable, hence availability is not affected. The impact is mainly on search performance and event ordering.

To address it, you can try the below: [Do it in Dev instances first and then to prod when applying settings]

First, review the data being collected and look for opportunities to reduce event size before ingestion (e.g., adjust Tenable export chunking) or use indexed extractions or field filtering to avoid massive raw events from scan results which is not meeting any use case or detection for you.

If the error still persists post reduction efforts, adjust the max_rawsize_perchunk setting in limits.conf. The default is 100 MB. if your events are larger, increase it conservatively (e.g., 200 MB) and monitor system performance. A common rule of thumb is to keep the value at no more than 10% of your index bucket size.

Please note that raising the limit clears the warning but can increase memory usage, so balance the setting against available resources.

The below reference would be helpful.
Ref: Events might not be returned in sub-second order due to search memory limits. | Splunk

>>
If this post addressed your question, you can:

  • Give it karma to show appreciation 👍
  • Mark it as the solution if it solved your issue ✔️
  • Add a comment if you’d like more details ✏️

Acknowledging helpful answers keeps the community strong and motivates contributors to continue sharing their expertise.
>>

View solution in original post

kknairr
Contributor

@Jtorge This error occurs because Splunk is under memory pressure when handling large events (scan outputs from tenable), so it may not return results in perfect sub‑second order. Your data is still indexed and searchable, hence availability is not affected. The impact is mainly on search performance and event ordering.

To address it, you can try the below: [Do it in Dev instances first and then to prod when applying settings]

First, review the data being collected and look for opportunities to reduce event size before ingestion (e.g., adjust Tenable export chunking) or use indexed extractions or field filtering to avoid massive raw events from scan results which is not meeting any use case or detection for you.

If the error still persists post reduction efforts, adjust the max_rawsize_perchunk setting in limits.conf. The default is 100 MB. if your events are larger, increase it conservatively (e.g., 200 MB) and monitor system performance. A common rule of thumb is to keep the value at no more than 10% of your index bucket size.

Please note that raising the limit clears the warning but can increase memory usage, so balance the setting against available resources.

The below reference would be helpful.
Ref: Events might not be returned in sub-second order due to search memory limits. | Splunk

>>
If this post addressed your question, you can:

  • Give it karma to show appreciation 👍
  • Mark it as the solution if it solved your issue ✔️
  • Add a comment if you’d like more details ✏️

Acknowledging helpful answers keeps the community strong and motivates contributors to continue sharing their expertise.
>>

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...