Splunk Search

How to use the REST API to just run a search and stream the results back?

a212830
Champion

Hi,

I have a question about using the REST API to run a search. The doc seems to indicate that you need to follow 3 steps - create a search job, get the search status, and then get the search results. Is there any way to just run the search and stream the results back? Seems like a lot of steps...

Tags (4)
1 Solution

neelamssantosh
Contributor

curl -ku username:password https://SearchHead_host:8089/servicesNS/admin/search/search/jobs/export -d search=“search index%3D_internal | head 3” -d output_mode=csv/xml/json

View solution in original post

venugoski
Explorer

Need help in the splunk api curl query, i am seeing the below error. 

curl -k -u apiuser:password "https://10.236.141.0:8089/services/search/jobs/export" -d search="search index=address-validation earliest=-15m latest=now source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" | stats dc(kubernetes.pod_name) as pod_count"

<?xml version="1.0" encoding="UTF-8"?>
<response>
<messages>
<msg type="ERROR">Unbalanced quotes.</msg>
</messages>
</response>



 

sometimes i dont see the result either:

curl -k -u user:password https://10.236.141.0:8089/services/search/jobs/export -d search="search index=address-validation earliest=-15m latest=now source=eventhub://sams-jupiter-prod-wus-logs-premium-1.servicebus.windows.net/address-validation; | stats dc(kubernetes.pod_name) as pod_count"



"<?xml version='1.0' encoding='UTF-8'?>

<results preview='0'>

<meta>

<fieldOrder />

</meta>

<messages>

  <msg type="INFO">Your timerange was substituted based on your search string</msg>

</messages>



</results>"
0 Karma

venugoski
Explorer

i am running the below query 

 

 

curl -k -u user:password https://10.236.142.0:8089/services/search/jobs/export -d search="search index=list-service source="eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;" "kubernetes.namespace_name"="list-service" | stats dc(kubernetes.pod_name) as pod_count"

<?xml version='1.0' encoding='UTF-8'?>
<results preview='0'>
<meta>
<fieldOrder />
</meta>
</results>
zsh: command not found: kubernetes.namespace_name=list-service | stats dc(kubernetes.pod_name) as pod_count

 

0 Karma

jwalthour
Communicator

In your search, you need to escape your quotes, like this:

 search="search index=list-service source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" \"kubernetes.namespace_name\"=\"list-service\" | stats dc(kubernetes.pod_name) as pod_count"

or use single quotes around the search contents:

search=‘search index=list-service source="eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;" "kubernetes.namespace_name”="list-service" | stats dc(kubernetes.pod_name) as pod_count’

 

0 Karma

venugoski
Explorer

Need help in the splunk api curl query, i am seeing the below error. 

curl -k -u apiuser:password "https://10.236.141.0:8089/services/search/jobs/export" -d search="search index=address-validation earliest=-15m latest=now source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" | stats dc(kubernetes.pod_name) as pod_count"

<?xml version="1.0" encoding="UTF-8"?>
<response>
<messages>
<msg type="ERROR">Unbalanced quotes.</msg>
</messages>
</response>


 

sometimes i dont see the result either:

curl -k -u user:password https://10.236.141.0:8089/services/search/jobs/export -d search="search index=address-validation earliest=-15m latest=now source=eventhub://sams-jupiter-prod-wus-logs-premium-1.servicebus.windows.net/address-validation; | stats dc(kubernetes.pod_name) as pod_count"

 

<?xml version='1.0' encoding='UTF-8'?>

<results preview='0'>

<meta>

<fieldOrder />

</meta>

<messages>

  <msg type="INFO">Your timerange was substituted based on your search string</msg>

</messages>

 

</results>

0 Karma

arunslal
Loves-to-Learn Lots

Looks like spaces and quotes are being identified as shell.  Try escaping them like below:

curl -k -u user:password https://10.236.142.0:8089/services/search/jobs/export -d search='search index=list-service source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" \"kubernetes.namespace_name\"=\"list-service\" | stats dc(kubernetes.pod_name) as pod_count'

I had a very long query that needed to be passed via rest api. I ran into such issues but url encoding the query was very helpful.

I used this website for that:

https://meyerweb.com/eric/tools/dencoder/

0 Karma

andrewlamonica
New Member

I wasn't able to get the above example to work. But, this one worked fine for me...

curl -k -u 'username:password' https://splunk.host.name.here:8089/services/search/jobs/export  -d search="search error | head 3" -d output_mode=xml

The main differences are...
1. I needed to quote my username and password (as they have special chars in them)
2. I needed to replace "servicesNS" with just "services"
3. Having "search" in the URL twice didn't work for me, I removed on of them.
4. Only one output_mode can be specified at a time (I put "xml" in my example, but the other two work, just not all at once)
5. I needed to remove the "smart quotes" and use normal quotes. That might just be my console being picky, though.

0 Karma

kutzi
Path Finder

I needed to add -d exec_mode=oneshot  otherwise it wouldn't stream the results back.

E.g.

curl -k -u 'username:password' https://splunk.host.name.here:8089/services/search/jobs/export  -d search="search index=_internal | head 3" -d output_mode=csv -d exec_mode=oneshot

adminpulse
Loves-to-Learn Lots

where should i run this command.

Also, please elaborate the command

curl -k -u 'username:password' https://splunk.host.name.here:8089/services/search/jobs/export  -d search="search index=_internal | head 3" -d output_mode=csv -d exec_mode=oneshot

 

Tags (1)
0 Karma

spunk311z
Path Finder

@kutzi  thank you SO MUCH for posting this!!   

I have spent several hours trying to figure out how to do a basic synchronous search via curl/api (have tried 100s of curl command variations).

I have scripts working with the Async method (as that is clearly documented in splunk docs),  however im not sure why the direct, synchronous method seems to have little/no documentation.  (i realize the pros/cons of each and that synchronous search should rarely be used).  

Again thanks for taking the time to make this post,  it was super helpful.

Here is what is working for me:

curl -u admin:mypw -k https://splunk.me:8089/services/search/jobs/export -d search="search index=routers Web Down | head 3" -d output_mode=csv -d exec_mode=oneshot

### also this works:
curl -u admin:mypw -k https://splunk.me:8089/services/search/jobs/export -d output_mode=json -d search="search index=routers |head 10"

 

neelamssantosh
Contributor

curl -ku username:password https://SearchHead_host:8089/servicesNS/admin/search/search/jobs/export -d search=“search index%3D_internal | head 3” -d output_mode=csv/xml/json

arunslal
Loves-to-Learn Lots

Hi @neelamssantosh  Apologies for asking as a comment. 

In case of multiserach, does the rest api call syntax differ? I'm able to do normal search with the above syntax but it fails when put a multisearch query instead. Can you please take a look? thank you.

0 Karma

a212830
Champion

Thanks! Works great.

0 Karma

neeldesai1992
Path Finder

the restful url call /serviceNS/admin/search/search/jobs/export is right?

0 Karma

neeldesai1992
Path Finder

Does this require to have saved search query? Or you are making it on demand?

0 Karma

nopslide
Engager

It's on demand

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...