All Apps and Add-ons

TA-webtools - CURL command error - schema not specified

splk
Communicator

Hello community and @jkat54 ,

I am currently testing your fancy webtools App.
It looks very promising, but i am running in an error I don’t understand.

Example:
(Notice, the csv simple gets me the id - i could also doe eval team_id=„12“)

index=test source=„NHL-Teams.csv“ Team=*Colorado* | eval team_id=ID | url_string= "https://statsapi.web.nhl.com/api/v1/teams/“.team_id | curl uri=url_string method=get debug=true | table curl*

gets me an „curl uri schema not specified“

| curl uri="https://statsapi.web.nhl.com/api/v1/teams/12" method=get debug=true | table curl*

is working as intended.

I can only suggest that this kind of string concatenation for building a url is not supported, but I dont understand why 🙂 Or do you suggest to do it in a different way?

Kind regards! 

Labels (1)
Tags (3)
0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

JSON, but you have to escape the quotes and the curly brackets as well if i recall correctly.

To use datafield and JSON for params as well, i think you need to set application/json header too.  Depends on your api.

However, i was able to make params work in uri=  havent tried in urifield, but probably needs escaping too.

View solution in original post

jkat54
SplunkTrust
SplunkTrust

When sending a field in as a uri variable, you must use urifield=fieldName instead.  We added that capability in 1.3.0

 

for sending payloads see this example:

 

  • Setting a Custom Header & Test Data:
    | makeresults count=1
    | eval header="{\"content-type\":\"application/json\"}"
    | eval data="{\"test data\":\"DATA\"}"
    | curl method=post uri=https://localhost:8089/services user=admin pass=changeme debug=true headerfield=header datafield=data

0 Karma

splk
Communicator

Hello,

ok, but in both examples I try to curl the uri directly (without payload), which is not working as expected.

 

Tags (1)
0 Karma

jkat54
SplunkTrust
SplunkTrust

In both examples you are adding parameters to the uri.  

instead I'm telling you to send them as a data field.

 

 

 

0 Karma

splk
Communicator

I previsouly tested the params with the data field, with no success.
So I tried the request with the linux CURL in putty, to make sure the request is working at all, with the result at the CLI it is working with the app not.

I dont now how to extend testing / debugging. Is there any additional logging? 

0 Karma

jkat54
SplunkTrust
SplunkTrust

Your first example works fine for me when i use yahoo.com.

 

 

Capture.PNG

0 Karma

jkat54
SplunkTrust
SplunkTrust

Are you on a Mac?

are you using "magic" quotes or straight quotes?

0 Karma

splk
Communicator

No - i am on a Windows machine (using Chrome) and the Splunk Server is on Linux.

Really strange - I get a html response thats something is odd with my authentication. I rechecked the api key with the CURL on CLI.

 

24-07-_2020_06-05-27.png

0 Karma

anthonymelita
Contributor

You can try your original method passing the team_id as a token to a |map command

index=test source="NHL-Teams.csv" Team=*Colorado* 
| eval team_id=ID 
| url_string= "https://statsapi.web.nhl.com/api/v1/teams/".team_id 
|map search="| curl uri=$url_string$ method=get debug=true"
| table curl*

 

0 Karma

jkat54
SplunkTrust
SplunkTrust

You don't need the map command

use urifield=fieldName instead 

0 Karma

splk
Communicator

I got one example running by removing ticks and quotes from the uri= field.

Still trying to get the datafield working - is the datafield json or params or both?

0 Karma

jkat54
SplunkTrust
SplunkTrust

JSON, but you have to escape the quotes and the curly brackets as well if i recall correctly.

To use datafield and JSON for params as well, i think you need to set application/json header too.  Depends on your api.

However, i was able to make params work in uri=  havent tried in urifield, but probably needs escaping too.

splk
Communicator

OK.

Maybe it would be helpful to upload the source to github for more contributions.
I would try to extend the documentation.

jkat54
SplunkTrust
SplunkTrust

Its found here:

https://github.com/bentleymi/Splunk/tree/master/TA-webtools

We'll take all the help we can get!

Many thanks for considering a contribution!!!

0 Karma

jkat54
SplunkTrust
SplunkTrust

I don't see anywhere in that response where it says your auth is bad.

must be further down.

thats way different from the original error you reported though. So it seems we've made progress.

0 Karma

splk
Communicator

Another example with virustotal:

 

works on CLI:
curl -X GET 'http://www.virustotal.com/vtapi/v2/domain/report?domain=tines.io&apikey=122555'

works NOT with webtools:
| curl method=get uri="http://www.virustotal.com/vtapi/v2/domain/report?domain=tines.io&apikey=123456" debug=true

 

Maybe I need some more examples in the documentation 🙂

0 Karma

jkat54
SplunkTrust
SplunkTrust

|makeresults count=1
| eval params="{\"domain\":\"Bob.com\",\"apikey\":\"key\"}"

| curl method=get uri="http://www.virustotal.com/vtapi/v2/domain/report debug=true datafield=params

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...