Need help with running Rest API to pull results of a Splunk query to a third party server

[Navanitha]

I am running a rest APi basically curl to query Splunk for results and export them to the server.  below is my api query.  My Splunk query is very big and the results are also kind of huge.  Query is running fine but I don't see any results.

#!/bin/bash

search_query=$(cat <<'EOF'
search index=my long splunk query
EOF
)

echo "Running Splunk search..."
curl --http1.1 -k -u admin:password \
"https://<splunk uri>:8089/services/search/jobs/export" \
--data-urlencode "search=$search_query" \
-d output_mode=csv \
-d earliest_time='-24d@d' \
-d latest_time='@d' \
-o output-file.csv

echo "Done. Results in output-file.csv"

This pi returns below results - 

curl: (18) transfer closed with outstanding read data remaining with empty ouput-file.csv.  Looks like it is not able to run such huge query.  I tried the curl command with some simple search query and it is working.  How can I make this work ?

[isoutamo]

Have you try to dispatch your query first then look when it’s ready and after that export results instead of do it in one operation?

[livehybrid]

Hi @Navanitha 

There are a number of things here which could be affecting this. Did you notice the time it took before crashing out? e.g. 1 minute, 5 minutes? There is a --max-time param you can pass to curl so Im wondering if this could help. 

Are you able to find the job in Splunk to see what its status is, how long it took to execute and how many results it returned? The easiest way to do this is probably via the Job manager (Top Right of Splunk screen under "Activity").

Is there a proxy/firewall between your machine and Splunk? Sometimes firewalls have a tendency to kill downloads or long-running HTTP calls.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing