Splunk Add-on Builder REST data input pagination

[yotamcp]

Hi,

I've recently started working with the Splunk Add-on Builder to quickly query and index data from a REST API of one of my systems.

The response looks something like this:

{
"count":296,
"next":"http://mysite.com/api/?page=2&page_size=200",
"previous":null,
"results": [...]
}

I was wondering if there is a way to paginate through the results.

Note: I am aware of checkpoints (I'm using this as well), but am looking for something else.

The behavior I'm looking for is a paginated query, that will continue until "next=null", and then sleep for the specified interval of the data input. 

Any ideas? (I would really prefer not implementing a whole new REST client, when there is already a really good implementation here)

Thanks!

[Tyler]

The REST data input is quite simple. You'll have to code in order to get it to pull more pages.

This would be a good feature request. I don't see it being difficult to implement.

[kfennell]

Working through the same issue here. I'm wondering if I have to script it myself so it loops through requests until there are no more results. Seems like something that would be common, so I am still thinking there must be configurable way....