topic Re: Properties/Arguments in Endpoint URL for REST Modular Input in Splunk Enterprise

Properties/Arguments in Endpoint URL for REST Modular Input

splunk_svc — Thu, 28 Sep 2017 04:55:00 GMT

Hi Splunkers.
I'm trying to set up a REST input to bring back output from an API.

These are the parameters used to form the API Endpoint URL. i.e.

https://<my_api_base_endpoint>/<1st_parameter>/<authtype>/token?api-version=<api_version>;

In the above example I need to provide the above parameters to build the entire endpoint URL both before and after the "?" etc as opposed to hard-coding them in the endpoint URL field in the setup screen.

The config screen in Splunk web config for the REST input provides an area for URL arguments and and HTTP Header properties but nothing used in either of these two areas seems to get substituted into the actual URL that Splunk calls when it tries to contact the endpoint.

Any advice on where these parameters go so they can flesh out the endpoint URL when it's called?
Note that the initial call to the API is a POST to get an access token with all subsequent calls being a GET.

Finally, in case it's relevant to answering the question, this input will be running on a HF.

Cheers and thanks in advance.

Re: Properties/Arguments in Endpoint URL for REST Modular Input

splunk_svc — Thu, 28 Sep 2017 05:19:53 GMT

BTW, we are running v6.6.3 of Splunk Enterprise.

Re: Properties/Arguments in Endpoint URL for REST Modular Input

Sukisen1981 — Tue, 29 Sep 2020 16:03:02 GMT

Hi,

You need to modify the bin/tokens.py UNDER the rest_ta app UNDE etc
Steps:
Configure REST Data Input
Endpoint URL: https:///<1st_parameter>//token?api-version=$z$ (taking z as an token example, api version 7)
URL Arguments: z=7
now navigate to bin/tokens.py UNDER the rest_ta app UNDE etc
add something like
def z():
z=7

return z

repeat same steps for token.
Try one dynamic parameter first..it will wok 🙂

Re: Properties/Arguments in Endpoint URL for REST Modular Input

Sukisen1981 — Thu, 28 Sep 2017 17:44:09 GMT

Custom Response Handlers
from Damien Dallimore himself - https://splunkbase.splunk.com/app/1546/#/details
You can provide your own custom Response Handler. This is a Python class that you should add to the
rest_ta/bin/responsehandlers.py module.
You can then declare this class name and any parameters in the REST Input setup page.
Such use cases for implementing a custom response handler might include
dynamic setting of URL arguments/HTTP header propertys ie: a stream cursor
extracting the next link to follow from the response content or HTTTP REL header
custom pre-processing of the raw HTTP response before indexing in Splunk
dynamically changing the REST endpoint URL

Re: Properties/Arguments in Endpoint URL for REST Modular Input

Damien_Dallimor — Fri, 29 Sep 2017 03:36:09 GMT

If I understand the flow correctly , you might need to use a custom Auth Handler that will :

1) check if you have an auth token yet
2) if false , then perform a POST to get an auth token to the Auth URL and save this token for future GET calls
3) if true , then proceed with a GET to your Data API URL with the saved token from step 2

Is this correct ? If so , can you respond with the format of your Auth URL and your Data API URL so I can give a more concrete configuration example ? Or is it simply the same URL from your example , just differentiated by it being a POST or a GET ?