topic Re: Using CURLP PHP to export results in Getting Data In

Using CURLP PHP to export results

maurelio79 — Tue, 29 Sep 2020 17:38:49 GMT

Hi to all, i'm trying to export result with php curl using this :

curl -u admin:changeme \
     -k https://localhost:8089/servicesNS/admin/search/jobs/1423855196.339/results/ \
     --get -d output_mode=csv-d count=5

I made different try in php, but i'm not able to pass parameter like output_mode and count.
If i try with CURLOPT_POST and CURLOPT_POSTFIELDS i get "Method Not Allowed"

Can someone help me please?

Thanks and regards.

Re: Using CURLP PHP to export results

mayurr98 — Sat, 13 Jan 2018 13:18:46 GMT

If i am not wrong there is space between output_mode=csv and -d count=5

Re: Using CURLP PHP to export results

maurelio79 — Sat, 13 Jan 2018 15:02:09 GMT

At the end i was able to do the php curl with a simple GET and calling

https://localhost:8089/services/search/jobs//results/?output_mode=csv&count=50000

Now the problem is that i get just 10.000 results (should be more than 18.000)

Re: Using CURLP PHP to export results

mayurr98 — Sat, 13 Jan 2018 15:19:01 GMT

Ohk whats is your search?
And do you get the same numberof results into splunk as well?

Re: Using CURLP PHP to export results

mayurr98 — Sat, 13 Jan 2018 15:25:54 GMT

Also set
this is from the limits.conf:

 [restapi]
 maxresultrows = <integer>
 * Maximum result rows to be returned by /events or /results getters from REST API.
 * Defaults to 50000.
As you can see, there is a limit configured.

You have two options now:

1) Enhance the limit to a value that is suitable for you.
2) I think the better option is to repeat your call with a different offset. You split up your requests on this way. Take a look into the answer of this post:

http://answers.splunk.com/answers/25411/upper-limit-for-rest-api-limits-conf-maxresultrows.html

Re: Using CURLP PHP to export results

maurelio79 — Sat, 13 Jan 2018 15:37:45 GMT

Default in limits.conf is 50000. Search is an inputlookup | table and it returns more than 18.000

Re: Using CURLP PHP to export results

maurelio79 — Tue, 16 Jan 2018 16:12:24 GMT

I write here complete answer if someone need.

As already wrote the correct query to curl and export results with PHP is:

https://localhost:8089/services/search/jobs//results/?output_mode=csv&count=50000

Then, the root cause of the results truncated to 10.000 was the sort in the table, solved with

| sort 0 <field>

Regards.

Re: Using CURLP PHP to export results

mayurr98 — Tue, 16 Jan 2018 16:22:46 GMT

Hey
Sort by default gives 10k results if you want unlimited results then you should use sort limit=0
I hope this helps you!