topic Re: How to use the REST API to just run a search and stream the results back? in Splunk Search

How to use the REST API to just run a search and stream the results back?

a212830 — Tue, 28 Apr 2015 12:51:47 GMT

Hi,

I have a question about using the REST API to run a search. The doc seems to indicate that you need to follow 3 steps - create a search job, get the search status, and then get the search results. Is there any way to just run the search and stream the results back? Seems like a lot of steps...

Re: How to use the REST API to just run a search and stream the results back?

neelamssantosh — Mon, 28 Sep 2020 19:41:49 GMT

curl -ku username:password https://SearchHead_host:8089/servicesNS/admin/search/search/jobs/export -d search=“search index%3D_internal | head 3” -d output_mode=csv/xml/json

Re: How to use the REST API to just run a search and stream the results back?

a212830 — Tue, 28 Apr 2015 21:13:51 GMT

Thanks! Works great.

Re: How to use the REST API to just run a search and stream the results back?

neeldesai1992 — Thu, 12 Oct 2017 17:54:06 GMT

Does this require to have saved search query? Or you are making it on demand?

Re: How to use the REST API to just run a search and stream the results back?

neeldesai1992 — Thu, 12 Oct 2017 20:03:22 GMT

the restful url call /serviceNS/admin/search/search/jobs/export is right?

Re: How to use the REST API to just run a search and stream the results back?

andrewlamonica — Mon, 17 Sep 2018 23:48:40 GMT

I wasn't able to get the above example to work. But, this one worked fine for me...

curl -k -u 'username:password' https://splunk.host.name.here:8089/services/search/jobs/export  -d search="search error | head 3" -d output_mode=xml

The main differences are...
1. I needed to quote my username and password (as they have special chars in them)
2. I needed to replace "servicesNS" with just "services"
3. Having "search" in the URL twice didn't work for me, I removed on of them.
4. Only one output_mode can be specified at a time (I put "xml" in my example, but the other two work, just not all at once)
5. I needed to remove the "smart quotes" and use normal quotes. That might just be my console being picky, though.

Re: How to use the REST API to just run a search and stream the results back?

kutzi — Tue, 28 Jul 2020 08:07:33 GMT

I needed to add -d exec_mode=oneshot otherwise it wouldn't stream the results back.

E.g.

curl -k -u 'username:password' https://splunk.host.name.here:8089/services/search/jobs/export -d search="search index=_internal | head 3" -d output_mode=csv -d exec_mode=oneshot

Re: How to use the REST API to just run a search and stream the results back?

spunk311z — Thu, 27 May 2021 17:57:38 GMT

@kutzi thank you SO MUCH for posting this!!

I have spent several hours trying to figure out how to do a basic synchronous search via curl/api (have tried 100s of curl command variations).

I have scripts working with the Async method (as that is clearly documented in splunk docs), however im not sure why the direct, synchronous method seems to have little/no documentation. (i realize the pros/cons of each and that synchronous search should rarely be used).

Again thanks for taking the time to make this post, it was super helpful.

Here is what is working for me:

curl -u admin:mypw -k https://splunk.me:8089/services/search/jobs/export -d search="search index=routers Web Down | head 3" -d output_mode=csv -d exec_mode=oneshot ### also this works: curl -u admin:mypw -k https://splunk.me:8089/services/search/jobs/export -d output_mode=json -d search="search index=routers |head 10"

Re: How to use the REST API to just run a search and stream the results back?

nopslide — Mon, 16 May 2022 15:03:58 GMT

It's on demand

Re: How to use the REST API to just run a search and stream the results back?

arunslal — Mon, 13 Mar 2023 14:22:53 GMT

Hi @neelamssantosh Apologies for asking as a comment.

In case of multiserach, does the rest api call syntax differ? I'm able to do normal search with the above syntax but it fails when put a multisearch query instead. Can you please take a look? thank you.

Re: How to use the REST API to just run a search and stream the results back?

adminpulse — Mon, 31 Jul 2023 11:50:13 GMT

where should i run this command.

Also, please elaborate the command

curl -k -u 'username:password' https://splunk.host.name.here:8089/services/search/jobs/export  -d search="search index=_internal | head 3" -d output_mode=csv -d exec_mode=oneshot

Re: How to use the REST API to just run a search and stream the results back?

venugoski — Thu, 19 Oct 2023 00:18:11 GMT

i am running the below query

curl -k -u user:password https://10.236.142.0:8089/services/search/jobs/export -d search="search index=list-service source="eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;" "kubernetes.namespace_name"="list-service" | stats dc(kubernetes.pod_name) as pod_count" <?xml version='1.0' encoding='UTF-8'?> <results preview='0'> <meta> <fieldOrder /> </meta> </results> zsh: command not found: kubernetes.namespace_name=list-service | stats dc(kubernetes.pod_name) as pod_count

Re: How to use the REST API to just run a search and stream the results back?

arunslal — Wed, 18 Oct 2023 20:08:23 GMT

Looks like spaces and quotes are being identified as shell. Try escaping them like below:

curl -k -u user:password https://10.236.142.0:8089/services/search/jobs/export -d search='search index=list-service source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" \"kubernetes.namespace_name\"=\"list-service\" | stats dc(kubernetes.pod_name) as pod_count'

I had a very long query that needed to be passed via rest api. I ran into such issues but url encoding the query was very helpful.

I used this website for that:

https://meyerweb.com/eric/tools/dencoder/

Re: How to use the REST API to just run a search and stream the results back?

venugoski — Thu, 26 Oct 2023 00:27:19 GMT

Need help in the splunk api curl query, i am seeing the below error.

curl -k -u apiuser:password "https://10.236.141.0:8089/services/search/jobs/export" -d search="search index=address-validation earliest=-15m latest=now source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" | stats dc(kubernetes.pod_name) as pod_count" <?xml version="1.0" encoding="UTF-8"?> <response> <messages> <msg type="ERROR">Unbalanced quotes.</msg> </messages> </response>

sometimes i dont see the result either:

curl -k -u user:password https://10.236.141.0:8089/services/search/jobs/export -d search="search index=address-validation earliest=-15m latest=now source=eventhub://sams-jupiter-prod-wus-logs-premium-1.servicebus.windows.net/address-validation; | stats dc(kubernetes.pod_name) as pod_count" "<?xml version='1.0' encoding='UTF-8'?> <results preview='0'> <meta> <fieldOrder /> </meta> <messages> <msg type="INFO">Your timerange was substituted based on your search string</msg> </messages> </results>"

Re: How to use the REST API to just run a search and stream the results back?

venugoski — Wed, 25 Oct 2023 18:30:40 GMT

Need help in the splunk api curl query, i am seeing the below error.

curl -k -u apiuser:password "https://10.236.141.0:8089/services/search/jobs/export" -d search="search index=address-validation earliest=-15m latest=now source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" | stats dc(kubernetes.pod_name) as pod_count"

<?xml version="1.0" encoding="UTF-8"?>
<response>
<messages>
<msg type="ERROR">Unbalanced quotes.</msg>
</messages>
</response>

sometimes i dont see the result either:

<?xml version='1.0' encoding='UTF-8'?>

<meta>

</meta>

<msg type="INFO">Your timerange was substituted based on your search string</msg>

</messages>

</results>

Re: How to use the REST API to just run a search and stream the results back?

jwalthour — Thu, 25 Jan 2024 07:28:39 GMT

In your search, you need to escape your quotes, like this:

search="search index=list-service source=\"eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;\" \"kubernetes.namespace_name\"=\"list-service\" | stats dc(kubernetes.pod_name) as pod_count"

or use single quotes around the search contents:

search=‘search index=list-service source="eventhub://sams-jupiter-prod-scus-logs-premium-1.servicebus.windows.net/list-service;" "kubernetes.namespace_name”="list-service" | stats dc(kubernetes.pod_name) as pod_count’