topic Export to csv is not fetching all the results - Python /Splunk SDK in Splunk Dev

Export to csv is not fetching all the results - Python /Splunk SDK

pchp348 — Tue, 29 Sep 2020 22:22:36 GMT

I do have a Python code to run an query and export the search results to .csv files. The program is working perfectly fine but when i opened the search results i could not see all the results. I just validated the same by running the query manually in splunk and exported the result and compared the results with the one which is generated through my code. I am running the query for last 2 hours.

My Code -
import time
import splunklib.client as client
import splunklib.results as results
import csv

import random

HOST = "Server"
PORT = 8089
USERNAME = "user"
PASSWORD = "password"

service = client.connect(
host=HOST,
port=PORT,
username=USERNAME,
password=PASSWORD)

My splunk query file

with open('H:\Query1.txt', 'r') as myfile:
Splunk_query=myfile.read()

Executing the query for last 2 hours

results_kwargs = {
"earliest_time": "-2h",
"latest_time": "now",
"search_mode": "normal",
"output_mode": "csv"
}

oneshotsearch_results = service.jobs.oneshot(Splunk_query, **results_kwargs)
f=open("H:\lasttwohours.csv", 'w')
f.write(oneshotsearch_results.read())
f.close()

Kindly help me with the export with the absolute results which i am getting it from splunk.

NOTE : I dont have permission to change any .conf file since this is the restricted environment. I can run my program and get the results from splunk.

Re: Export to csv is not fetching all the results - Python /Splunk SDK

lesley_lin — Thu, 17 Oct 2019 03:42:33 GMT

http://dev.splunk.com/view/python-sdk/SP-CAAAEE5
The document says:

By default, one-shot searches will return a maximum of 100 events, even if there are more than 100 events in the search results. To return more than 100 events, add the following parameter to your one-shot search's arguments:
"count": 0
The count parameter, when set to zero, indicates that there is no limit to the number of events to be returned.

Maybe you could try to modify your search query.

Re: Export to csv is not fetching all the results - Python /Splunk SDK

lesley_lin — Thu, 17 Oct 2019 06:11:06 GMT

Like this:
job = service.jobs.oneshot(searchquery, **results_kwargs, count=0)