How to save memory and ignore table row where column value is less than x?

[firstname]

I may use a search similar to this:

index=mock_index source=mock_source
| eval event = _raw
| stats count as frequency by event
| table event, frequency

which results in a table similar to the one below:

Event	Frequency
2022-08-22 13:11:12 [stuff] apple.bean.34 [stuff]	2000
2022-08-22 14:18:22 [stuff] apple.bean.86 6 [stuff]	200
2022-08-22 15:17:42 [stuff] apple.bean.1 546 [stuff]	2

Some of the tables which I get from this search give an error that states the search_process_memory_usage_threshold has been exceeded.  If I know that I am not interested in rows where the frequency is less than 1,000, is there a way to limit the table so it only shows the rows above 1,000?  Would this also improve memory usage?

[firstname]

This helped me in limiting the number of rows in the table, but it doesn't seem to impact memory usage.  To reduce memory usage, I had to use a more specific search than just:

index=mock_index source=mock_source

 Giving this an extra keyword reduced the memory usage.

[richgalloway · Accepted solution]

You can filter by frequency, but it will be after the stats command, which probably is where the memory is being used up.  Try eliminating unneeded fields.  That can save a lot of memory.

index=mock_index source=mock_source
| fields _raw - *
| eval event = _raw
| stats count as frequency by event
| where frequency > 1000
| table event, frequency

[firstname]

The line below helped in limiting the number of rows in the table.

| where frequency > 1000

However, this line below did not seem to impact memory usage.

| fields _raw - *

I chose to add an extra search parameter.  This seemed to reduce memory usage.  Instead of this:

index=mock_index source=mock_source

I did this:

index=mock_index source=mock_source mock_parameter