topic Re: REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession? in Getting Data In

REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

ks2211 — Tue, 05 Jan 2016 19:19:53 GMT

Hi All,

I am using the Splunk REST API (mainly search, savedsearch endpoints) to get data out of Splunk.

Currently I am trying to do the following:

Create a saved search
Dispatch said search to get SID
Check status of the job with given SID
Get the results of the job for SID back

Right now, I have steps 1,2, and 4 working fine.
I can run steps 1 and 2 in succession without issues. Step 3 I can run right after 1 and 2, but having issues.
Step 3, I can run it, but having issues here because it seems like I need to poll to get the status of the Job--is there a better way to handle this (mainly to check the status of the job)?
Step 4, I can run in isolation AFTER i have the SID, but cannot run 1,2,3,4 in succession.

Any suggestions on fixing step 3? I need to check the status and only continue when it is "DONE" but can't figure out a way to keep checking the status.

Re: REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

ks2211 — Tue, 05 Jan 2016 19:52:11 GMT

I can't edit my post for Karma points reasons but I had one correction :

Step 3, I can run it but having issues here because it seems like I need to poll to get the status of the Job--is there a better way to handle this (mainly to check the status of the job)--I found a workaround where I do a loop and do a check on the job over an interval (in golang's terms, use time.Sleep at this section). Ideally i'd like some sort of trigger that lets this function finish when either "DONE" or "FAILED" is returned from the job status.

Re: REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

jplumsdaine22 — Wed, 06 Jan 2016 17:33:49 GMT

Save yourself pain and just use export. See: http://docs.splunk.com/Documentation/Splunk/6.3.2/RESTREF/RESTsearch#search.2Fjobs.2Fexport

Otherwise as you have found you'll have to do it manually, by polling at a specific interval. Use while or a recursive if statement.

Re: REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

ks2211 — Wed, 06 Jan 2016 17:40:14 GMT

Thanks for your reply. Right now, I'm using a pretty hackish method and doing a loop and checking the value

How exactly does export work? Does it just give back all of the data for a particular search?

Thanks

Re: REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

jplumsdaine22 — Wed, 06 Jan 2016 17:48:01 GMT

The results are streamed back to you. Try doing the example curl: http://docs.splunk.com/Documentation/Splunk/6.3.1511/RESTREF/RESTsearchExamples#search.2Fjobs.2Fexport_GET

For slow searches this may tie up your thread until the search is complete, but if you're doing small searches anyway its probably more efficient.

Re: REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

ks2211 — Wed, 06 Jan 2016 18:04:14 GMT

Got it, thanks! This may be my alternative solution because I'm trying to get the results by SID where this requires the search query.

I'll have to look into some of the features of Golang and figure out if theres a way to do this.

Thanks!