TA-webtools - CURL command error - schema not specified

[splk]

Hello community and @jkat54 ,

I am currently testing your fancy webtools App.
It looks very promising, but i am running in an error I don’t understand.

Example:
(Notice, the csv simple gets me the id - i could also doe eval team_id=„12“)

index=test source=„NHL-Teams.csv“ Team=*Colorado* | eval team_id=ID | url_string= "https://statsapi.web.nhl.com/api/v1/teams/“.team_id | curl uri=url_string method=get debug=true | table curl*

gets me an „curl uri schema not specified“

| curl uri="https://statsapi.web.nhl.com/api/v1/teams/12" method=get debug=true | table curl*

is working as intended.

I can only suggest that this kind of string concatenation for building a url is not supported, but I dont understand why 🙂 Or do you suggest to do it in a different way?

Kind regards! 

[jkat54]

When sending a field in as a uri variable, you must use urifield=fieldName instead.  We added that capability in 1.3.0

for sending payloads see this example:

• Setting a Custom Header & Test Data:
  | makeresults count=1
  | eval header="{\"content-type\":\"application/json\"}"
  | eval data="{\"test data\":\"DATA\"}"
  | curl method=post uri=https://localhost:8089/services user=admin pass=changeme debug=true headerfield=header datafield=data

[splk]

Hello,

ok, but in both examples I try to curl the uri directly (without payload), which is not working as expected.

[jkat54]

In both examples you are adding parameters to the uri.  

instead I'm telling you to send them as a data field.

[splk]

I previsouly tested the params with the data field, with no success.
So I tried the request with the linux CURL in putty, to make sure the request is working at all, with the result at the CLI it is working with the app not.

I dont now how to extend testing / debugging. Is there any additional logging? 

[jkat54]

Your first example works fine for me when i use yahoo.com.

[jkat54]

Are you on a Mac?

are you using "magic" quotes or straight quotes?

[splk]

No - i am on a Windows machine (using Chrome) and the Splunk Server is on Linux.

Really strange - I get a html response thats something is odd with my authentication. I rechecked the api key with the CURL on CLI.

[anthonymelita]

You can try your original method passing the team_id as a token to a |map command

index=test source="NHL-Teams.csv" Team=*Colorado* 
| eval team_id=ID 
| url_string= "https://statsapi.web.nhl.com/api/v1/teams/".team_id 
|map search="| curl uri=$url_string$ method=get debug=true"
| table curl*

[jkat54]

You don't need the map command

use urifield=fieldName instead 

[splk]

I got one example running by removing ticks and quotes from the uri= field.

Still trying to get the datafield working - is the datafield json or params or both?

[jkat54 · Accepted solution]

JSON, but you have to escape the quotes and the curly brackets as well if i recall correctly.

To use datafield and JSON for params as well, i think you need to set application/json header too.  Depends on your api.

However, i was able to make params work in uri=  havent tried in urifield, but probably needs escaping too.

[splk]

OK.

Maybe it would be helpful to upload the source to github for more contributions.
I would try to extend the documentation.

[jkat54]

Its found here:

https://github.com/bentleymi/Splunk/tree/master/TA-webtools

We'll take all the help we can get!

Many thanks for considering a contribution!!!

[jkat54]

I don't see anywhere in that response where it says your auth is bad.

must be further down.

thats way different from the original error you reported though. So it seems we've made progress.

[splk]

Another example with virustotal:

works on CLI:
curl -X GET 'http://www.virustotal.com/vtapi/v2/domain/report?domain=tines.io&apikey=122555'

works NOT with webtools:
| curl method=get uri="http://www.virustotal.com/vtapi/v2/domain/report?domain=tines.io&apikey=123456" debug=true

Maybe I need some more examples in the documentation 🙂

[jkat54]

|makeresults count=1
| eval params="{\"domain\":\"Bob.com\",\"apikey\":\"key\"}"

| curl method=get uri="http://www.virustotal.com/vtapi/v2/domain/report debug=true datafield=params