REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

[ks2211]

Hi All,

I am using the Splunk REST API (mainly search, savedsearch endpoints) to get data out of Splunk.

Currently I am trying to do the following:

1. Create a saved search
2. Dispatch said search to get SID
3. Check status of the job with given SID
4. Get the results of the job for SID back

Right now, I have steps 1,2, and 4 working fine.
I can run steps 1 and 2 in succession without issues. Step 3 I can run right after 1 and 2, but having issues.
Step 3, I can run it, but having issues here because it seems like I need to poll to get the status of the Job--is there a better way to handle this (mainly to check the status of the job)?
Step 4, I can run in isolation AFTER i have the SID, but cannot run 1,2,3,4 in succession.

Any suggestions on fixing step 3? I need to check the status and only continue when it is "DONE" but can't figure out a way to keep checking the status.

[jplumsdaine22 · Accepted solution]

Save yourself pain and just use export. See: http://docs.splunk.com/Documentation/Splunk/6.3.2/RESTREF/RESTsearch#search.2Fjobs.2Fexport

Otherwise as you have found you'll have to do it manually, by polling at a specific interval. Use while or a recursive if statement.

[ks2211]

Thanks for your reply. Right now, I'm using a pretty hackish method and doing a loop and checking the value

How exactly does export work? Does it just give back all of the data for a particular search?

Thanks

[jplumsdaine22]

The results are streamed back to you. Try doing the example curl: http://docs.splunk.com/Documentation/Splunk/6.3.1511/RESTREF/RESTsearchExamples#search.2Fjobs.2Fexpo...

For slow searches this may tie up your thread until the search is complete, but if you're doing small searches anyway its probably more efficient.

[ks2211]

Got it, thanks! This may be my alternative solution because I'm trying to get the results by SID where this requires the search query.

I'll have to look into some of the features of Golang and figure out if theres a way to do this.

Thanks!

[ks2211]

I can't edit my post for Karma points reasons but I had one correction :

Step 3, I can run it but having issues here because it seems like I need to poll to get the status of the Job--is there a better way to handle this (mainly to check the status of the job)--I found a workaround where I do a loop and do a check on the job over an interval (in golang's terms, use time.Sleep at this section). Ideally i'd like some sort of trigger that lets this function finish when either "DONE" or "FAILED" is returned from the job status.