See title, I'm using a scheduled query to prune a set of results from a lookup table, this lookup table has over 2m results, but after the prune, it's truncated down to 50,000. This exclusively happens when I schedule the lookup table with the "replace" option. Append works perfectly.
Pruning script:
| inputlookup my_lookup.csv
| where _time > relative_time(now(),"-6m")
Pruning schedule options:
I've tried setting the output location to both my_lookup.csv and to other lookups. In both cases, 50,000 results seems to be the limit for the replaced lookup table.
Append schedule options:
Any help is appreciated.
Hi @Mr_person,
You are hitting the max_action_results on limits.conf which is default 50000. It is not a good idea to increase this limit to 2 million.
It seems your only option is using outputlookup in your search although it seems not safe.
limits.conf
[scheduler]
max_action_results = <integer>
* The maximum number of results to load when triggering an alert action.
* Default: 50000
Hi @Mr_person,
You are hitting the max_action_results on limits.conf which is default 50000. It is not a good idea to increase this limit to 2 million.
It seems your only option is using outputlookup in your search although it seems not safe.
limits.conf
[scheduler]
max_action_results = <integer>
* The maximum number of results to load when triggering an alert action.
* Default: 50000
Also, I'm aware of the fact that I could change the report to use the outputlookup command inside of the search string, but that makes the report unsafe for others to open. For those looking for a workaround:
| inputlookup my_lookup.csv
| where _time > relative_time(now(),"-6m")
| outputlookup my_lookup.csv append=false
Again this is unsafe and not ideal though.