Using version 1.2.6 of WebTools, I am running into into an issue where POSTs with request bodies are being rejected by the service I am calling due to invalid/mutilated JSON. When I run the exact same queries against WebTools version 1.2.2, they work fine. I see some subtle differences in the app's bin\curl.py file that I think are responsible. Here is the high-level format of my query: | makeresults | eval header="{\"Content-Type\":\"application/json\"}" | eval payload="(...)" | curl method=post ssl=true user=(...) pass=(...) uri="(...)" datafield=payload headerfield=header debug=true (...) I include the "debug=true" to demonstrate a slight difference between 1.2.2 and 1.2.6/. Under 1.2.6, a payload like this... { "query": { "bool": { "must": [ {"term": (...) ... results in the 'curl_data_payload' output of: { "query": { "bool": { "must": [ {"term": (...) However, under 1.2.6, the same payload results in the 'curl_data_payload' output of: { u'query': { u'bool': { u'must': [ {u'term': (...) which makes sense when I look at the bin\curl.py file and find this line: data = json.loads(result[options['datafield']]) I played around with bin\curl.py until I could get things to work. If I take this text: # STREAMING Use Case: iterate through results and run curl commands if len(results) > 0: for result in results: # use JSON encoded header string if provided if 'headerfield' in options: headers = json.loads(result[options['headerfield']]) else: headers = None # if data in options, set data = options['data'] if 'data' in options: data = str(options['data']) # if datafield in options, set datafield = options['datafield'] if 'datafield' in options: try: data = json.loads(result[options['datafield']]) except: data = str(result[options['datafield']]) else: data = None # debugging option if 'debug' in options: if options['debug'].lower() in ("yes", "true", "t", "1"): # for debugging we add results which show the options \ # that were sent to the curl command result['curl_method'] = method result['curl_verifyssl'] = verifyssl result['curl_uri'] = uri result['curl_splunkauth'] = splunkauth if data != None: result['curl_data_payload'] = data if headers: result['curl_header'] = headers # based on method, execute appropriate function if method.lower() in ("get","g"): Result = get(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) if method.lower() in ("head","h"): Result = head(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) if method.lower() in ("post","p"): Result = post(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) if method.lower() in ("put"): Result = put(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) if method.lower() in ("delete","del","d"): Result = delete(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) ... and make a few adjustments: # STREAMING Use Case: iterate through results and run curl commands if len(results) > 0: for result in results: # use JSON encoded header string if provided if 'headerfield' in options: headers = json.loads(result[options['headerfield']]) else: headers = None # if data in options, set data = options['data'] if 'data' in options: data = str(options['data']) # if datafield in options, set datafield = options['datafield'] if 'datafield' in options: data = str(options['datafield']) else: data = None # debugging option if 'debug' in options: if options['debug'].lower() in ("yes", "true", "t", "1"): # for debugging we add results which show the options \ # that were sent to the curl command result['curl_method'] = method result['curl_verifyssl'] = verifyssl result['curl_uri'] = uri result['curl_splunkauth'] = splunkauth if data != None: result['curl_data_payload'] = result[data] if headers: result['curl_header'] = headers # based on method, execute appropriate function if method.lower() in ("get","g"): Result = get(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) if method.lower() in ("head","h"): Result = head(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) if method.lower() in ("post","p"): Result = post(uri,sessionKey,verifyssl,headers,result[data],user,passwd,timeout) if method.lower() in ("put"): Result = put(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) if method.lower() in ("delete","del","d"): Result = delete(uri,sessionKey,verifyssl,headers,data,user,passwd,timeout) The queries will work once I make those adjustments. To be honest, I don't know enough about Python to explain why my adjustments fix things. I basically just looked at how 1.2.2 did things and adjusted the 1.2.6 to match. Is anybody else running into cases where POSTs with a request body are rejected after updating WebTools to 1.2.6? Splunk version: 7.1.1 'requests' python module version: 2.6.0 python version: 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] ... View more

Using the latest version (1.2.0) of WebTools, I am running into an issue where I can’t do a cURL against an API using authentication provided in the header itself. WebTools appears to be forcing Basic authentication (unless Splunk authentication is specified, in which case it builds a custom header). To demonstrate the issue, here is a vanilla cURL: In this case, the API is complaining that that I forgot the X-Sysdig-Product header. If I do the same request in Web Tools, I get a 401 Unauthorized instead of a 400 Bad Request: Based on what I see in this app's curl.py file, basic authentication is always going to be sent to the "request" library get() method (unless Splunk authorization is specified; then the get call in the 'else' block is selected). I would suggest that "def get(..." in curl.py is modified so that the requests.get() call does not send "auth=(user,password)", if both the user and password inputs are set to the default "None". try: if sessionKey == None: if user ==None and password == None: r = requests.get(uri,data=payload,verify=verifyssl, headers=headers, timeout=timeout) else: r = requests.get(uri,auth=(user,password),data=payload,verify=verifyssl, headers=headers, timeout=timeout) else: (…) ... View more