topic 100 result limit in JS SDK in Splunk Search

100 result limit in JS SDK

LordVoldemort — Wed, 11 Jul 2012 23:26:49 GMT

I tried adding "count" to params object when calling service.search() but it doesn't work. How do I get more than 100 results from the api when using the javascript SDK?

Re: 100 result limit in JS SDK

ChrisG — Mon, 28 Sep 2020 12:03:50 GMT

I am far from expert on this. But I can quote some information I have from an expert:

The maximum number of results is always going to be the value in max_count (which you can set when you create the search). The maximum number of events is going to be max_count * status_buckets, which you can also set when you create the search.

Re: 100 result limit in JS SDK

ineeman — Thu, 12 Jul 2012 01:07:54 GMT

Great question. There are a couple of different limits on how to get more data from a Splunk search, but I'll only go over the important ones here.

When you create a search, you can specify how many status buckets this search can have, and how many events/results should be stored per status bucket. The default is 10,000 per status bucket, and the default number of status buckets is 0. So if you did a search with:

service.search("query", function(...) { ... })

The maximum number of results you can get back is 10,000. If you instead did:

service.search("query", {status_buckets: 300}, function(...) { ... })

You would get a maximum of 300 * 10,000 results stored. Note that 300 is the default value used by the UI. You could also pass in something like:

{max_count: 50000}

and that will similarly change things.

However, even with setting max_count/status_buckets, the default number of results you get back when you do something like:

mySearchJob.results({}, function(err, results) { ... })

is going to be 100, per the REST API documentation (here). You can pass in a count parameter to override this, as such:

mySearchJob.results({count: 500}, function(err, results) { ... })

and this will fetch 500 results at a time. You can also paginate through the results by using a combination of count and offset, e.g.:

mySearchJob.results({count: 500, offset: 1000}, function(err, results) { ... })

Finally, the maximum number of results you can get in a single call to the search/jobs/{sid}/results endpoint (which is what mySearchJob.results(...) is calling under the hood) is defined by the maxresultsrows value in limits.conf (in the restapi stanza). This is by default 50,000. Note that you can retrieve more than 50,000 results from a single search, you just have to do it using multiple API calls (i.e. pagination). You can see a sample of this here.

Hopefully this makes sense. If not, let me know, and I'll clarify some more! 🙂

Re: 100 result limit in JS SDK

LordVoldemort — Thu, 12 Jul 2012 16:40:54 GMT

Thanks for the thorough response.

In this case, putting count in the call to job.results() was all I needed.

Re: 100 result limit in JS SDK

sideview — Fri, 20 Jul 2012 03:13:22 GMT

I think these comments about status buckets only apply if the search has no transforming commands. Status buckets are relevant really only to the streaming portion of the search. Just fyi.