All Apps and Add-ons

Rest API Modular Input: Combining multiple REST URLs?

Communicator

I am trying to pull volume data in from my VMAX array. I have about 200 volumes and I need a unique REST call for each one.

The call looks like this:

https://unisphere:8443/univmax/restapi/provisioning/symmetrix/XXXXXXXXXX/volume/<VOLUMEID>;

Is there a way to programmatically populate the <VOLUMEID> while iterating call through a list? I know you can do $foo$ substitution, but it seems to me like that is for single instances, not multiple.

Thanks!

1 Solution

Ultra Champion

The latest release of the REST API Modular Input v1.3.7 , now supports tokens that can return a list of values.

Token replacement functions in the URL can return a list of values that will cause multiple URL's to be formed and the requests for these URL's will be executed in parallel in multiple threads.

See tokens.py for an example.

So for your use case above just create a function in tokens.py that returns a list of volume id's

def volumeids():
    return ['123','456','789']

And then setup your URL like :

https://unisphere:8443/univmax/restapi/provisioning/symmetrix/XXXXXXXXXX/volume/$volumeids$

View solution in original post

Ultra Champion

The latest release of the REST API Modular Input v1.3.7 , now supports tokens that can return a list of values.

Token replacement functions in the URL can return a list of values that will cause multiple URL's to be formed and the requests for these URL's will be executed in parallel in multiple threads.

See tokens.py for an example.

So for your use case above just create a function in tokens.py that returns a list of volume id's

def volumeids():
    return ['123','456','789']

And then setup your URL like :

https://unisphere:8443/univmax/restapi/provisioning/symmetrix/XXXXXXXXXX/volume/$volumeids$

View solution in original post

Contributor

This is awesome, just came across this problem and its already solved!

Communicator

Aaaaaaaand this is why the splunk community is the best community. Thanks Damien!

Ultra Champion

I aim to please 🙂

0 Karma

Ultra Champion

As you correctly note , you can do a substitution on a single URL , but with the current code this would still require you to setup 200 different REST input stanzas.

Your best bet might be to fork a copy of the code on Github and create your own "multiplexed" hack for your specific needs.
Should only require very minimal code.

Perhaps you could add some code around the "replaceTokens" method in rest.py to check if the token substitution function from tokens.py returns 1 or more results , and if it returns multiple results , then perform multiple URL requests in a loop rather than just 1 single request.

0 Karma

Ultra Champion

Update : I got the itch to code and I am working on this for you.Hang tight.

Communicator

Thanks for that! I think it will open up a ton of flexibility for the input.

0 Karma

New Member

hi damien, does that mean you'll be able to make multiple substitutions on a single url? would that enable a kind of cartesian product?

0 Karma

Ultra Champion

Correct.Multiple requests will get spawned in parallel or optionally sequentially.If you have multiple tokens in your URL that return lists , then "Cartesianing" will occur. But all responses to these requests will still get indexed as individual events in Splunk.

Trivial example :

Lets say you create these 2 token functions :

def sometoken():
    return ['goo','foo']

def anothertoken():
    return ['boo','hoo']

Will spawn 2 requests :

http://foo.com?param1=$sometoken$

Will spawn 4 requests :

http://foo.com?param1=$sometoken$&param2=$anothertoken$
0 Karma