What I want to do is pull down the results of a saved search as a CSV file, which will then be loaded into a data warehouse. I would like to automate this on a nightly basis, so I'm experimenting with the web api.
I've successfully been able to pull down the results of the search I want via the web API, but am running into two issues which I have not found info about in the docs:
When I browse through the results with the web interface, I get 2600 results. However, I'm only getting about 200 in the csv I'm pulling down. How do I get the rest of the result set?
Can I somehow set up the saved search so I only get the fields I am interested in, rather than the other 100 fields I don't care about?
I assume that you're using the REST API like:
GET /services/search/jobs/<sid>/results?output_mode=csv
If this is the case, there are two arguments to add, field_list
and count
:
GET /services/search/jobs/<sid>/results?output_mode=csv&field_list=<comma separated field list>&count=0
If you want to page through the results for efficiency, you can use the offset
argument in addition to the count
argument and make several requests.
ps: is this info in the docs somewhere?
This does just what I need - thanks!