Getting Data In
Highlighted

Why is REST API removing a leading pipe before an "inputcsv" command?

Communicator

It appears that my use of the REST API is somehow causing a leading pipe to be stripped before an inputcsv command. I have this python search string:

 "| inputcsv scale_med_validation_data | apply fastflux_model | where 'predicted(is_attack)' = 1 | eval t = now()+3600*1 | eval report_hour=strftime(t, "%H") | eval report_date=strftime(t, "%m/%d/%Y") | tail 50 | collect index=fastflux_summary"

This works as desired when entered manually through the web interface.

However, when submitted through the REST API, the jobs screen shows the search query missing the leading pipe:

"inputcsv scale_med_validation_data | apply fastflux_model | where 'predicted(is_attack)' = 1 | eval t = now()+3600*1 | eval report_hour=strftime(t, "%H") | eval report_date=strftime(t, "%m/%d/%Y") | tail 50 | collect index=fastflux_summary"

Naturally, this causes the inputcsv to fail, and so none of the REST API jobs succeed. Why might the leading pipe not be making it through here?

Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Splunk Employee
Splunk Employee

Hi @kcnolan13,
What endpoint are you using to submit the search?

Have you tried escaping the pipe character?

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Communicator

My base URL is https://xx.xx.xx.xx:8089/

What method of escaping are you referring to? I tried sticking a "\" in front of the leading pipe, but only ended up with a parse error.

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Splunk Employee
Splunk Employee

Ok, it looks like you are using the correct management port to submit the request. But what endpoint are you using to submit the search? Are you creating a saved search and then retrieving the results? Are you using an SDK or is there anything else about how you are submitting the search that might help troubleshoot?

It might be good to get more context before going further with escaping characters. That might not be the issue.

For extensive troubleshooting, it might also be helpful to contact support.

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Communicator

I'm using a nearly identical Python script to the example shown here:

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Communicator

The important part probably being:

sid = httplib2.Http(disablesslcertificate_validation=True).request(baseurl + '/services/search/jobs','POST',
headers={'Authorization': 'Splunk %s' % sessionKey},body=urllib.urlencode({'search': searchQuery}))[1]

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Splunk Employee
Splunk Employee

Thanks for the info. I have an active request in to our engineering team to review the Python example here and will add your question/issue to this.

In the meantime, in case it is possible to consider alternatives, there is a Python SDK for developers that might be helpful to you, with info on creating + running searches here:
http://dev.splunk.com/view/python-sdk/SP-CAAAEE5

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Communicator

Thanks @frobinson. I'm aware of the SDK, but hoped I could just bang out this small task with a modified version of the example Python script. I hope the developers fix this issue, if it is indeed on their end.

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Splunk Employee
Splunk Employee

I understand. I've pinged some folks again about this, will post again here if I get an update. Sorry for the confusion!

0 Karma
Highlighted

Re: Why is REST API removing a leading pipe before an "inputcsv" command?

Path Finder

Try this:

"search | inputcsv ..."
0 Karma