Running a dbxquery through jobs.export my results are limited to 100k rows. Do I need to paginate streaming results?
Here's my code:
data = {
'adhoc_search_level': 'fast',
'search_mode': 'normal',
'preview': False,
'max_count': 500000,
'output_mode': 'json',
'auto_cancel': 300,
'count': 0
}
job = service.jobs.export(<dbxquery>, **data)
reader = results.JSONResultsReader(job)
lst = [result for result in reader if isinstance(result, dict)]
This runs correctly except that that results are always stopped at 100k rows, it should be over 200k.
... View more