In trying to speed up queries by using buffered export API in the python SDK as discussed here, running into a problem where the service jobs.oneshot query works, while the jobs.export version fails with ParseError .
The query parameters are otherwise the same.
Moreover the location of the parse error varies each time the query is run (date/time parameters are fixed so it is presumably getting the same data each time).
By looping through the generator, ie using results.ResultsReader().next() , and trapping Exceptions and returning python type() , I see that the query seems to work for an initial segment of data
in other words, the returned OrderedDict objects are ok -
but that there is a XML parse errors that occurs at different records each time I run the query (recall the data extracted should be the same each time)
0 <class 'splunklib.results.Message'>
1 <class 'splunklib.results.Message'>
2 <class 'splunklib.results.Message'>
3 <class 'collections.OrderedDict'>
4 <class 'collections.OrderedDict'>
5 <class 'collections.OrderedDict'>
...
4044 <class 'xml.etree.ElementTree.ParseError'>
(rerunning this, the record # will vary)
Here's a sample of the Traceback:
Traceback (most recent call last):
File "/Users/zk8n1ue/miniconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-153-517a13bba7fa>", line 1, in <module>
tmp3 = [parsefunc(x) for x in tq3['results']]
File "<ipython-input-153-517a13bba7fa>", line 1, in <listcomp>
tmp3 = [parsefunc(x) for x in tq3['results']]
File "/Users/zk8n1ue/miniconda3/lib/python3.7/site-packages/splunklib/results.py", line 210, in next
return next(self._gen)
File "/Users/zk8n1ue/miniconda3/lib/python3.7/site-packages/splunklib/results.py", line 219, in _parse_results
for event, elem in et.iterparse(stream, events=('start', 'end')):
File "/Users/zk8n1ue/miniconda3/lib/python3.7/xml/etree/ElementTree.py", line 1222, in iterator
yield from pullparser.read_events()
File "/Users/zk8n1ue/miniconda3/lib/python3.7/xml/etree/ElementTree.py", line 1297, in read_events
raise event
File "/Users/zk8n1ue/miniconda3/lib/python3.7/xml/etree/ElementTree.py", line 1269, in feed
self._parser.feed(data)
File "<string>", line unknown
ParseError: not well-formed (invalid token): line 454841, column 54713
Rerunning, yields the same trace except for the location of invalid token. eg
ParseError: not well-formed (invalid token): line 169194, column 13753
ParseError: not well-formed (invalid token): line 204476, column 30137
Any ideas what could be causing this and is there a workaround?
By manually
... View more