Solved: Why is REST API removing a leading pipe before an ... - Page 2

kcnolan13 · ‎01-26-2017

It appears that my use of the REST API is somehow causing a leading pipe to be stripped before an inputcsv command. I have this python search string:

 "| inputcsv scale_med_validation_data | apply fastflux_model | where 'predicted(is_attack)' = 1 | eval t = now()+3600*1 | eval report_hour=strftime(t, "%H") | eval report_date=strftime(t, "%m/%d/%Y") | tail 50 | collect index=fastflux_summary"

This works as desired when entered manually through the web interface.

However, when submitted through the REST API, the jobs screen shows the search query missing the leading pipe:

"inputcsv scale_med_validation_data | apply fastflux_model | where 'predicted(is_attack)' = 1 | eval t = now()+3600*1 | eval report_hour=strftime(t, "%H") | eval report_date=strftime(t, "%m/%d/%Y") | tail 50 | collect index=fastflux_summary"

Naturally, this causes the inputcsv to fail, and so none of the REST API jobs succeed. Why might the leading pipe not be making it through here?

frobinson_splun · ‎01-27-2017

Hey @kcnolan13,
I just heard back from our engineering team and there is an issue with the script as shown in the docs. Specifically, the issue is where it checks for queries starting with 'search' and then prepends 'search' if it's not found. Here is an updated script that should fix the problem. Note this update here:

" # If the query doesn't already start with the 'search' operator or another
# generating command (e.g. "| inputcsv"), then prepend "search " to it.
if not (searchQuery.startswith('search') or searchQuery.startswith("|")):
searchQuery = 'search ' + searchQuery"
I will update the docs. Let me know how this works for you!

import urllib
import httplib2
from xml.dom import minidom

baseurl = 'https://re-latitude.sv.splunk.com:8089'
userName = 'guest'
password = 'guest'

searchQuery = '| inputcsv foo.csv | where sourcetype=access_common | head 5'

# Authenticate with server.
# Disable SSL cert validation. Splunk certs are self-signed.
serverContent = httplib2.Http(disable_ssl_certificate_validation=True).request(baseurl + '/services/auth/login',
    'POST', headers={}, body=urllib.urlencode({'username':userName, 'password':password}))[1]

sessionKey = minidom.parseString(serverContent).getElementsByTagName('sessionKey')[0].childNodes[0].nodeValue

# Remove leading and trailing whitespace from the search
searchQuery = searchQuery.strip()

# If the query doesn't already start with the 'search' operator or another 
# generating command (e.g. "| inputcsv"), then prepend "search " to it.
if not (searchQuery.startswith('search') or searchQuery.startswith("|")):
    searchQuery = 'search ' + searchQuery

print searchQuery

# Run the search.
# Again, disable SSL cert validation. 
print httplib2.Http(disable_ssl_certificate_validation=True).request(baseurl + '/services/search/jobs','POST',
    headers={'Authorization': 'Splunk %s' % sessionKey},body=urllib.urlencode({'search': searchQuery}))[1]

View solution in original post

Why is REST API removing a leading pipe before an "inputcsv" command?

Announcing Scheduled Export GA for Dashboard Studio

Extending Observability Content to Splunk Cloud

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!