I need to call a saved search using resultsReader class using the python splunk sdk. The result set of the saved search is greater than 50K events.
I might not be using correctly the API or there is a bug in the API. The problem I am facing is that after some events are returned I am getting the following exception in results.py.
Traceback (most recent call last):
File "mysavedsearch.py", line 64, in <module>
for re in results.ResultsReader(blocksearch_results):
File "/opt/splunk/bin/scripts/splunk-sdk-python-1.2.2/examples/../splunklib/results.py", line 205, in next
return self._gen.next()
File "/opt/splunk/bin/scripts/splunk-sdk-python-1.2.2/examples/../splunklib/results.py", line 258, in _parse_results
except et.ParseError as pe:
AttributeError: 'module' object has no attribute 'ParseError'
This is my code:
import urllib2,sys,csv,results,json
from time import sleep
import splunklib.client as client
import splunklib.results as results
HOST = "hello.com"
PORT = 8089
USERNAME = "user_id"
PASSWORD = "mypwd"
OWNER = "owner"
APP = "appname"
# Create a Service instance and log in
service = client.connect(
host=HOST,
port=PORT,
username=USERNAME,
password=PASSWORD,
app=APP,
owner=OWNER)
mysavedsearch = service.saved_searches["saved_search_name"]
# Run the saved search
job = mysavedsearch.dispatch()
# Wait for the job to finish
while True:
job.refresh()
stats = {"isDone": job["isDone"],
"doneProgress": float(job["doneProgress"])*100,
"scanCount": int(job["scanCount"]),
"eventCount": int(job["eventCount"]),
"resultCount": int(job["resultCount"])}
status = ("\r%(doneProgress)03.1f%% %(scanCount)d scanned "
"%(eventCount)d matched %(resultCount)d results") % stats
if stats["isDone"] == "1":
break
count = 100
offset = 0
while (offset < int(resultCount)):
kwargs_paginate = {"count": count,
"offset": offset}
# Get the search results
blocksearch_results = job.results(**kwargs_paginate)
for re in results.ResultsReader(blocksearch_results):
print re
# Increase the offset to get the next set of results
offset += count
job.cancel()
Thanks,
Lp
Note:
The issue has been fixed in the next Python SDK version.
Hello lpolo,
The line of code you're hitting in results.py is masking the problem you've encountered. It's the result of a bug in the Python SDK. We will release a fix to this issue soon.
I'd also like to understand the problem you've encountered and address it. Can you do the following?
At file "splunklib/results.py", line 258, in _parse_results, change
except et.ParseError as pe:
except et.ParseError as pe:
to:
except SyntaxError as pe:
Rerun your script.
Send me two things: The traceback and--if possible--the saved search you're running or--if that's not possible--some other search that produces the same result.
Thanks,
David Noble
1) Line 258 changed.
2) Script returned:
Traceback (most recent call last):
File "feed_xcal_undefined_topN_daily.py.back", line 64, in
for re in results.ResultsReader(blocksearch_results):
File "/opt/splunk/bin/scripts/splunk-sdk-python-1.2.2/examples/../splunklib/results.py", line 205, in next
return self._gen.next()
File "/opt/splunk/bin/scripts/splunk-sdk-python-1.2.2/examples/../splunklib/results.py", line 249, in _parse_results
values.append(elem.text.encode('utf8'))
AttributeError: 'NoneType' object has no attribute 'encode'
3) Query pasted in case 160361.
HI @lpolo.
We've looked in to this and it is indeed a bug. We're going to fix it and we'll get it out there.
Thanks for reporting this and for using the SDK!
Glenn
Hi @lpolo
Can you give us some details on the environment.
Thanks!
1) python2.6
2) I tried with the latest SDK "splunk-sdk-python-1.2.2" and the same problem.
3) RedHat. 2.6.32-358.23.2.el6.x86_64 #1 SMP Wed Oct 16 18:37:12 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Thanks,
Lp