Getting Data In

REST API Python - Issue with pulling results before search job is done.

sd248011
New Member

I wrote a script in Python to run a search query and return the results. The code to send the search query is:

sid1 = httplib2.Http(disable_ssl_certificate_validation=True).request(baseurl + '/services/search/jobs','POST',
headers={'Authorization': 'Splunk %s' % sessionKey},body=urllib.urlencode({'search': searchQuery1}))[1]

The code to return the results is:

response1 = httplib2.Http(disable_ssl_certificate_validation=True).request(baseurl + '/services/search/jobs/' + slicesid1 + '/results?count=0','GET',
headers={'Authorization': 'Splunk %s' % sessionKey},body=urllib.urlencode({'search': searchQuery1}))[1]

The issue is that if the results request is made before the result query is completed, no results will be returned. I have fiddled with sleep time which allows me to return results but that isn't very efficient as the sleep time can be too long or too short.

I know there is a field called dispatchState that is RUNNING during the search which changes to DONE when it is completed. How can I put some code in after the initial search query that will continuously check to see if the job is RUNNING or DONE and then once DONE, the results query will run?

Tags (1)
0 Karma
1 Solution

Neeraj_Luthra
Splunk Employee
Splunk Employee

You can write a while loop and continue to refresh the job and check for isDone property on the job. Exit when the property value changes to 1.

Checkout the Python SDK and code sample for this while loop at http://dev.splunk.com/view/SP-CAAAEE5#normaljob.

View solution in original post

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

You can write a while loop and continue to refresh the job and check for isDone property on the job. Exit when the property value changes to 1.

Checkout the Python SDK and code sample for this while loop at http://dev.splunk.com/view/SP-CAAAEE5#normaljob.

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

How about this ...

kwargs_results = {"count": 0}
search_results = job.results(**kwargs_results)

0 Karma

sd248011
New Member

I was able to use the count argument in my original code at the top and pull all of the results. Now I am using the code in the link you gave me with the python SDK and can't figure out how to add in the count=0 argument. In your link:

for result2 in results.ResultsReader(job.results()):
print result2

The above code is to pull the results now.

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

Try this:

body=urllib.urlencode({'search': searchQuery1, 'count': 0})

0 Karma

sd248011
New Member

Thanks much that is exactly what I needed. Only problem I have now is it is not returning all of my results. Can you let me know how I would add in the count=0 argument on the results line that is:

for result2 in results.ResultsReader(job.results()):
print result2

0 Karma
Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...