All Apps and Add-ons

Splunk Add-on Builder - REST API input pagination

marco1987
Explorer

Hello splunkers,

I'm working with the latest version of Splunk Add-on Builder to index data from a REST API.

TA only pulls the first page of results by calling:

 

https://mywebpage.com/api/source/v2

 

At the bottom of the pulled data are URL for the next url:

 

"next_url" : "/api/source/v2?last=5431"

 

How do I configure TA for iterates through all the pages?

I checked from link below, but i dont' understand how (or if is possible) pass the variable  from modular input to my endpoint like this or in other way:

 

https://mywebpage.com/api/source/v2?last=${next_url}

 

https://docs.splunk.com/Documentation/AddonBuilder/4.3.0/UserGuide/ConfigureDataCollection#Pass_valu... 

Any ideas?

Thanks!

Labels (2)
0 Karma

marnall
Motivator

I recommend coding your modular input. You can use the page number as a checkpoint, then index a page and increment or decrement the checkpoint. Set an interval so that your input gets a page every X seconds, then has a condition to stop when the checkpoint gets to the end. 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...