Getting Data In

REST API: Create Search, Dispatch, Get Status, and Results. How can I run this flow in succession?

ks2211
Engager

Hi All,

I am using the Splunk REST API (mainly search, savedsearch endpoints) to get data out of Splunk.

Currently I am trying to do the following:

  1. Create a saved search
  2. Dispatch said search to get SID
  3. Check status of the job with given SID
  4. Get the results of the job for SID back

Right now, I have steps 1,2, and 4 working fine.
I can run steps 1 and 2 in succession without issues. Step 3 I can run right after 1 and 2, but having issues.
Step 3, I can run it, but having issues here because it seems like I need to poll to get the status of the Job--is there a better way to handle this (mainly to check the status of the job)?
Step 4, I can run in isolation AFTER i have the SID, but cannot run 1,2,3,4 in succession.

Any suggestions on fixing step 3? I need to check the status and only continue when it is "DONE" but can't figure out a way to keep checking the status.

0 Karma
1 Solution

jplumsdaine22
Influencer

Save yourself pain and just use export. See: http://docs.splunk.com/Documentation/Splunk/6.3.2/RESTREF/RESTsearch#search.2Fjobs.2Fexport

Otherwise as you have found you'll have to do it manually, by polling at a specific interval. Use while or a recursive if statement.

View solution in original post

0 Karma

jplumsdaine22
Influencer

Save yourself pain and just use export. See: http://docs.splunk.com/Documentation/Splunk/6.3.2/RESTREF/RESTsearch#search.2Fjobs.2Fexport

Otherwise as you have found you'll have to do it manually, by polling at a specific interval. Use while or a recursive if statement.

0 Karma

ks2211
Engager

Thanks for your reply. Right now, I'm using a pretty hackish method and doing a loop and checking the value

How exactly does export work? Does it just give back all of the data for a particular search?

Thanks

0 Karma

jplumsdaine22
Influencer

The results are streamed back to you. Try doing the example curl: http://docs.splunk.com/Documentation/Splunk/6.3.1511/RESTREF/RESTsearchExamples#search.2Fjobs.2Fexpo...

For slow searches this may tie up your thread until the search is complete, but if you're doing small searches anyway its probably more efficient.

0 Karma

ks2211
Engager

Got it, thanks! This may be my alternative solution because I'm trying to get the results by SID where this requires the search query.

I'll have to look into some of the features of Golang and figure out if theres a way to do this.

Thanks!

0 Karma

ks2211
Engager

I can't edit my post for Karma points reasons but I had one correction :

Step 3, I can run it but having issues here because it seems like I need to poll to get the status of the Job--is there a better way to handle this (mainly to check the status of the job)--I found a workaround where I do a loop and do a check on the job over an interval (in golang's terms, use time.Sleep at this section). Ideally i'd like some sort of trigger that lets this function finish when either "DONE" or "FAILED" is returned from the job status.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...