This question: How to use IN function with VALUE-LIST as a search or lookup  discusses using IN for a single key and list of values. Can that approach be generalized for lists of  KV lists?  Want to abstract what could be done in a verbose way with  AND and OR's : (keyA=value1 AND keyB=value2) OR (keyA=value3 AND keyB=value4) OR (keyA=value5 AND keyB=value6)...           ... View more

In trying to speed up queries by using buffered export API in the python SDK as discussed here, running into a problem where the service jobs.oneshot query works, while the jobs.export version fails with ParseError . The query parameters are otherwise the same. Moreover the location of the parse error varies each time the query is run (date/time parameters are fixed so it is presumably getting the same data each time). By looping through the generator, ie using results.ResultsReader().next() , and trapping Exceptions and returning python type() , I see that the query seems to work for an initial segment of data in other words, the returned OrderedDict objects are ok - but that there is a XML parse errors that occurs at different records each time I run the query (recall the data extracted should be the same each time) 0 <class 'splunklib.results.Message'> 1 <class 'splunklib.results.Message'> 2 <class 'splunklib.results.Message'> 3 <class 'collections.OrderedDict'> 4 <class 'collections.OrderedDict'> 5 <class 'collections.OrderedDict'> ... 4044 <class 'xml.etree.ElementTree.ParseError'> (rerunning this, the record # will vary) Here's a sample of the Traceback: Traceback (most recent call last): File "/Users/zk8n1ue/miniconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "<ipython-input-153-517a13bba7fa>", line 1, in <module> tmp3 = [parsefunc(x) for x in tq3['results']] File "<ipython-input-153-517a13bba7fa>", line 1, in <listcomp> tmp3 = [parsefunc(x) for x in tq3['results']] File "/Users/zk8n1ue/miniconda3/lib/python3.7/site-packages/splunklib/results.py", line 210, in next return next(self._gen) File "/Users/zk8n1ue/miniconda3/lib/python3.7/site-packages/splunklib/results.py", line 219, in _parse_results for event, elem in et.iterparse(stream, events=('start', 'end')): File "/Users/zk8n1ue/miniconda3/lib/python3.7/xml/etree/ElementTree.py", line 1222, in iterator yield from pullparser.read_events() File "/Users/zk8n1ue/miniconda3/lib/python3.7/xml/etree/ElementTree.py", line 1297, in read_events raise event File "/Users/zk8n1ue/miniconda3/lib/python3.7/xml/etree/ElementTree.py", line 1269, in feed self._parser.feed(data) File "<string>", line unknown ParseError: not well-formed (invalid token): line 454841, column 54713 Rerunning, yields the same trace except for the location of invalid token. eg ParseError: not well-formed (invalid token): line 169194, column 13753 ParseError: not well-formed (invalid token): line 204476, column 30137 Any ideas what could be causing this and is there a workaround? By manually ... View more

Is it possible, via Splunk's Python SDK, to specify event sampling ratio (say 1:1000) or some equivalent random evaluation in a subsearch which returns a long OR expression while specifying that the outer search does not sample? For concreteness, the subsearch is: [search index=my_index | rex "(?i)deviceId=(?P<DevId>[^ ]+)" | dedup DevId | return 1000000 $DevId] This returns a long OR lists, each of which can match one or more events. It is critical to extract all events associated with the randomly sampled device. ... View more

I'd like to (1) use a subquery to extract a list of deviceId's then (2) search the same index for all events containing any of those devices returned by the subquery. However, format puts quotes around each deviceId value only: deviceId="abc123" rather than around the equal sign: "deviceId=abc123" . Consequently the outer search doesn't match any events, while the latter modified form does. Is there an option for format to adjust quotes accordingly? Concrete example (1) index=myIndex DeviceLog | rex "(?i)deviceId=(?P<DevId>[^ ]+)" | stats values(DevId) as deviceId | format returns a list of the form: ( ( ( deviceId="0002ac61d" OR deviceId="0003511e" ... OR deviceId="0006ecff" ) ) ) But the query/subquery combination doesn't match any events: index=myIndex DeviceLog [search index=myIndex DeviceLog | rex "(?i)deviceId=(?P<DevId>[^ ]+)" | stats values(DevId) as deviceId | format] I've also tried a subquery variation using return like this: ...| dedup DevId | return 100000 $DevId] this almost works because it matches the deviceId values but doesn't match the key prefix deviceId= which can result in false positives (cookies caching device id's in different parts of the device log) ... View more

In a search executed via Python SDK, the stat list truncates results to 100 results, despite the fact that count=0. Is there another config or variable that controls stats to remove this limit? ... View more