topic Re: REST API Modular Input v1.3.7 - Enhancements in All Apps and Add-ons

REST API Modular Input v1.3.7 - Enhancements

DanielFordWA — Wed, 08 Jul 2015 12:06:29 GMT

I have started to learn python and made some basic edits to the REST API Modular Input v1.3.7 tokens.py and responsehandlers.py files.

There is some extra functionality I think would be very helpful.

It would be great if the polling interval could be done on a CRON schedule. I only need to query some endpoints once a day and entering a 86400 second polling interval only seems to start from the enabling of the endpoint. I am being asked to set the time for 3am, waking up at 3am to setup the Data Input is a bit difficult. Also when people setup inputs how can I find out when the data is due to arrive? I may be wrong on this, sorry if I am.
Love the recent functionality that tokens can return a list of values. http://answers.splunk.com/answers/260134/rest-api-modular-input-combining-multiple-rest-url-1.html However on some backloads I have been asked to leave a certain amount of time between each call as I am back loading up to 2 years worth of data.

Thanks,

Dan

Re: REST API Modular Input v1.3.7 - Enhancements

Damien_Dallimor — Wed, 08 Jul 2015 14:42:56 GMT

Cheers , I'll look into this as soon as I can.

Re: REST API Modular Input v1.3.7 - Enhancements

DanielFordWA — Fri, 10 Jul 2015 13:31:36 GMT

I've noticed that if the Splunk server is restarted the REST Data Input will pull back data even with a 8600 delay.

The delay does not seem to carry over after the restart.

Adding a CRON Schedule I guess would solve this issue.

Thanks for looking at the enhancements

Re: REST API Modular Input v1.3.7 - Enhancements

Damien_Dallimor — Fri, 10 Jul 2015 13:36:45 GMT

I have zero idea what the specification is for the REST endpoints you are polling.
But perhaps (and I am 100% guessing here) in the documentation there are options in the arguments for the REST endpoint you are calling that you can use to maintain a cursoring position. It might be an id or timestamp based cursor for example (based off other REST services such as Twitter's that I have seen).

Re: REST API Modular Input v1.3.7 - Enhancements

Damien_Dallimor — Tue, 14 Jul 2015 03:24:55 GMT

UPDATE : code written , need to test a bit before release.

Re: REST API Modular Input v1.3.7 - Enhancements

DanielFordWA — Tue, 14 Jul 2015 07:30:14 GMT

Thanks. The setup I work with is a bit different from most, lots of the application is built on REST endpoints. This will be of great help.

Re: REST API Modular Input v1.3.7 - Enhancements

Damien_Dallimor — Wed, 15 Jul 2015 16:12:31 GMT

The latest release , version 1.3.9, of the REST API Modular Input now has this functionality. Enjoy !

Re: REST API Modular Input v1.3.7 - Enhancements

DanielFordWA — Wed, 15 Jul 2015 16:15:07 GMT

Thanks for this. Much appreciated

Re: REST API Modular Input v1.3.7 - Enhancements

DanielFordWA — Thu, 16 Jul 2015 15:49:37 GMT

The enhancements work a treat! It makes a huge difference to how I can use Splunk, much appreciated.

Re: REST API Modular Input v1.3.7 - Enhancements

Damien_Dallimor — Thu, 16 Jul 2015 16:06:45 GMT

Good to hear 🙂