Splunk Search

Help with an API one shot search time discrepancy

tonymorin
Explorer

I see significant search time discrepancy when I run a one-shot search via the python SDK as opposed to when I run the same search in Splunk web with the same user, has anyone one else seen this? I assume I'm doing something wrong.
I am using the basic code form the example found: http://dev.splunk.com/view/python-sdk/SP-CAAAEE5
The only changes I added were in the kwargs_normalsearch I added earliest_time": "-60m,
"latest_time": "now".

It works just fine, but it is super slow. Me running the search in the GUI 46 seconds. Me running the search via REST API, 5 minutes plus. Not sure what the issues are???

Any guidance will be appreciated, Thanks in advance.

CODE:

import sys
from time import sleep
import splunklib.results as results

 ...

Initialize your service like so
import splunklib.client as client
service = client.connect(username="admin", password="yourpassword")

searchquery_normal = "search * | head 10"
kwargs_normalsearch = {"exec_mode": "normal"}
job = service.jobs.create(searchquery_normal, **kwargs_normalsearch)

A normal search returns the job's SID right away, so we need to poll for completion

While True:

  while not job.is_ready():
        pass
    stats = {"isDone": job["isDone"],
             "doneProgress": float(job["doneProgress"])*100,
              "scanCount": int(job["scanCount"]),
              "eventCount": int(job["eventCount"]),
              "resultCount": int(job["resultCount"])}

    status = ("\r%(doneProgress)03.1f%%   %(scanCount)d scanned   "
              "%(eventCount)d matched   %(resultCount)d results") % stats

    sys.stdout.write(status)
    sys.stdout.flush()
    if stats["isDone"] == "1":
        sys.stdout.write("\n\nDone!\n\n")
        break
    sleep(2)

# Get the results and display them
for result in results.ResultsReader(job.results()):
    print result

job.cancel()   
sys.stdout.write('\n')

SEARCH:

searchquery_normal = '''| tstats summariesonly=t chunk_size=10000 count first(All_Traffic.action) as action first(All_Traffic.icmp_type) as icmp_type from datamodel=Network_Traffic by _time span=1s All_Traffic.dest_ip All_Traffic.src_ip All_Traffic.dest_port All_Traffic.src_port All_Traffic.transport  | `drop_dm_object_name("All_Traffic")`  | eval external_src=if((!cidrmatch("10.0.0.0/8",src_ip) AND !cidrmatch("REDACTED/12",src_ip) and !cidrmatch("192.168.0.0/16",src_ip)),"true", "false")  | eval external_dest=if((!cidrmatch("10.0.0.0/8",dest_ip) AND !cidrmatch("REDACTED/12",dest_ip) and !cidrmatch("192.168.0.0/16",dest_ip)),"true", "false")  | eval external_dest=if(cidrmatch("REDACTED/16",dest_ip) AND external_src="false", "false", external_dest)  | eval external_dest=if(cidrmatch("REDACTED",dest_ip) AND cidrmatch("REDACTED/16",src_ip), "false", external_dest)  | search external_dest="true" OR external_src="true"  | fields - external_*'''.format(value_to_check, value_to_check)
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...