Splunk Dev

Know CSV file size before download

brunoabreu
Loves-to-Learn Lots

Using the Splunk Python SDK, is there any way to know the size of the CSV file that will be generated after streaming and writing all the results?

I've managed to achieve this with the approach below, but it makes me to download and iterate the CSV lines two times.

splunk_job = service.jobs.create(query, **kwargs)

# waits job to be done

splunk_job_result_args = {
    "output_mode": "csv"
}

splunk_job_results = splunk_job.results(**splunk_job_result_args)
results_length = 0
for bytes_csv_line in splunk_job_results:
    results_length += len(bytes_csv_line)
splunk_job_results.close()

# checks if the results_length exceeds the limit
# if not, executes the following:

splunk_job_results = splunk_job.results(**splunk_job_result_args)
for bytes_csv_line in splunk_job_results:
    # writes the bytes in a file
splunk_job_results.close()

 

Labels (2)
Tags (5)
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...