I'm the developer of the R Project app and currently working on issue #13.
When executing this...
index=_internal | r "output=data.frame(count=nrow(input))"
... it returns a count column with one row containing the number of events that are passed from the search (which is implicitly a search command) to the r command. But it's limited to the maxresultrows setting.
Based on what I see limits.conf documentation, that's the expected behaviour.
However, when executing this...
index=_internal | stats count
.. it returns the actual event count (which is not limited!), to me that's an unexpected behaviour.
Why is the number of search results not limited when passing piping the events to a stats command?
The stats command is a streaming command that uses map reduce to return the results of its calculations.
In the case of
index=_internal | stats count
only one actual result is returned - count
The search query may run against millions of actual events, but the events are not returned by the search, just the result.
The maxresults setting limits the number of results returned by the whole search, not the number of events it can scan.
The stats command is a streaming command that uses map reduce to return the results of its calculations.
In the case of
index=_internal | stats count
only one actual result is returned - count
The search query may run against millions of actual events, but the events are not returned by the search, just the result.
The maxresults setting limits the number of results returned by the whole search, not the number of events it can scan.