I want to use splunklib to run a one-off Splunk query and save it to csv.
I'm testing with a small query (a single visitId) of 8 events only.
The result is returned immediately in Splunk UI but I have problems getting the result from the python-sdk.
My problems with splunklib are:
- service.jobs.export() query does not complete because it keeps repeating the same 8 event results over and over again
- service.jobs.oneshot() query does not finish and returns no result
I tried adding the search parameters "preview"=False, i.e.
kwargs_export = { "search_mode": "normal","preview": True }
rr = results.ResultsReader(service.jobs.export(query,**kwargs_export ))
The only effect is that neither option returns anything anymore, since the queries are not completing.
import splunklib.client as client
import splunklib.results as results
service = client.connect(
host=HOST,
port=8089,
username=USERNAME,
password=PWD )
query= """search index=xxx application="xxx" sourcetype=xxx|
spath visitId | join type ..."""
rr = results.ResultsReader(service.jobs.export(query))
for item in rr:
for key in item.keys():
print(key, len(item[key]), item[key])
I tried the same with oneshot
kwargs_oneshot = {'output_mode': 'csv',"search_mode": "normal"}
oneshotsearch_results = service.jobs.oneshot(query, **kwargs_oneshot)
f=open('myresults.csv', 'w')
f.write(oneshotsearch_results.read())
This creates a csv file but has no content at all. I think .read is deprecated.
Any suggestions ?
All I want is to save the query results to .csv ONCE using the library.
Thanks!
Hello @to914868,
please add f.close()
on the next line after f.write(oneshotsearch_results.read())
I think the content is not flushed to the file.
@to914868, please accet this answer in order for other users to find more easily the answer to this question.
Hello @to914868,
please add f.close()
on the next line after f.write(oneshotsearch_results.read())
I think the content is not flushed to the file.
@to914868, please accet this answer in order for other users to find more easily the answer to this question.
Thanks @poete!
Here is what I used in the end
results_kwargs = {
"earliest_time": "-40min",
"latest_time": "now",
"search_mode": "normal",
"output_mode": "csv"
}
oneshotsearch_results = service.jobs.oneshot(query, **results_kwargs)
f=open('myresults.csv', 'w')
f.write(oneshotsearch_results.read())
f.close()
This is working fine , But i could not fetch all the results in csv. Kindly provide me the solution for this question
https://answers.splunk.com/answers/708529/export-to-csv-is-not-fetching-all-the-results-pyth.html?mi...
try:
kwargs_export = { "output_mode": "csv"}
rr = service.jobs.export(query)
for item in rr:
print(item)
I think that you shouldn't need to convert the result into resultsreader because it already is one.
This works.