Splunk Search

Why does the Splunk Java SDK always return 500k results, but I get 800k results in Splunk Web?

DataWarehousing
Explorer

The job returns 800k results in Splunk Web, whereas the Java API always returns 500k.

ifotopoulos
Explorer

Even though that seems like an old question let me give you my 2 cents since I encountered the same problem.

Let me start by clarifying that this question is not about the maxresultsrows that is set by default to 50k, the op is already paginating through the results and he is getting them in chunks of 50k.
The problem here is another limit of 500k. (notice the extra 0)

max_count = <integer>
* The number of events that can be accessible in any given status bucket 
  (when status_buckets = 0).
* The last accessible event in a call that takes a base and count.
* Note: This value does not reflect the number of events displayed in the 
  UI after the search is evaluated or computed.
* Default: 500000

You have to change that limit.

bontesl
Explorer

Perhaps a year late, but this is the answer that works for us

0 Karma

lguinn2
Legend

This is set in limits.conf - here is the documentation from limits.conf.spec

[restapi]
maxresultrows = <integer>
* Maximum result rows to be returned by /events or /results getters from REST
  API.
* Defaults to 50000.

Also answered on Limited results when running searches via REST API

Create a limits.conf file in $SPLUNK_HOME/etc/system/local. Change the value and then restart Splunk to have the new value take effect.

DataWarehousing
Explorer

I am already paginating through my results. Should it not help me get all the records irrespective of the maxResultRows value ?

0 Karma

lguinn2
Legend

No. The GUI is not constrained by this rule. Accessing search results via the REST API is constrained by this rule.

0 Karma

somesoni2
Revered Legend

Seems like the maxresultrow is set to 500K. Have a look at following post to how to retrieve larger results.

http://dev.splunk.com/view/java-sdk/SP-CAAAEPZ#paginating

0 Karma

DataWarehousing
Explorer

No it is not set to 500k. And i am already paginating through results. Still my upper cap remains to be 500k. When i inspect the job created by the API hit on the GUI it also shows the ResultCount to be 500k. But still when i search using the same query on the SPLUNK GUI it returns me 800k results.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!