How to search with join returns result in GUI, but not with Python SDK?

adomenico — Thu, 07 Jul 2022 20:02:15 GMT

I have a search that joins an index to a .csv lookup. When I run the search for last 24 hours in the GUI, I get ~81k matches (expected). When I run the exact same query via the sdk, I get 0 matches. Here is my code:

service = client.connect(
host=HOST,
port=PORT,
username=USERNAME,
password=PASSWORD)

import sys
from time import sleep
import splunklib.results as results

query= "search index=my_index sourcetype=my_sourcetype | fields field1 field2 field3 field4 field5 field6 field7 | join my_primary_key[| inputlookup my_lookup_file.csv ]"
kwargs = {"exec_mode": "normal",
"earliest_time": "-1440m",
"latest_time": "now",
"search_mode": "normal",
"output_mode": "json"
}
job = service.jobs.create(query, **kwargs)

# A normal search returns the job's SID right away, so we need to poll for completion
while True:
while not job.is_ready():
pass
stats = {"isDone": job["isDone"],
"doneProgress": float(job["doneProgress"])*100,
"scanCount": int(job["scanCount"]),
"eventCount": int(job["eventCount"]),
"resultCount": int(job["resultCount"])}

status = ("\r%(doneProgress)03.1f%% %(scanCount)d scanned "
"%(eventCount)d matched %(resultCount)d results") % stats

sys.stdout.write(status)
sys.stdout.flush()
if stats["isDone"] == "1":
sys.stdout.write("\n\nDone!\n\n")
break
sleep(2)

# Get the results and display them
for result in results.JSONResultsReader(job.results(output_mode='json')):
print(result)

job.cancel()
sys.stdout.write('\n')

Can somebody please explain why the query would work and return matches in the GUI but not via the SDK?

Re: How to search with join returns result in GUI, but not with Python SDK?

PickleRick — Thu, 07 Jul 2022 20:38:12 GMT

Do you use the same user in GUI as with REST?

BTW, join over inputlookup doesn't seem to be the best idea. Why not straight use lookup?

Re: How to search with join returns result in GUI, but not with Python SDK?

adomenico — Fri, 08 Jul 2022 13:23:43 GMT

Yes, it is the same user. I need to join, because the lookup csv file contains part of the data I need in the final report and the index has the other part.

topic Re: How to search with join returns result in GUI, but not with Python SDK? in Splunk Dev

How to search with join returns result in GUI, but not with Python SDK?

Re: How to search with join returns result in GUI, but not with Python SDK?

Re: How to search with join returns result in GUI, but not with Python SDK?