Splunk Dev

real-time search using python SDK export command

esharf
Engager

so here is my code:

 

import splunklib.client as client
import splunklib.results as results

client.connect(**connection_args)
job_kwargs = {"search_mode": "realtime", "earliest_time": "rt", "latest_time": "rt"}
for item in service.jobs.export(query=my_query, **job_kwargs):
    if isinstance(item, results.Message):
        print(item.message)
    else:
        print(item)

 

when I'm trying to run this code with a general query

 

query="search index=main"

 

It’s working properly.
but if I’m trying with

 

query="search `notable` | eval rule_name=if(isnull(rule_name),source,rule_name) | eval rule_title=if(isnull(rule_title),rule_name,rule_title) | `get_urgency` | `risk_correlation` | eval rule_description=if(isnull(rule_description),source,rule_description) | eval security_domain=if(isnull(security_domain),source,security_domain)"

 

I get a lot of events that I cannot see in the regular search.
also, I get almost every multiple times with a little change (such as dest_ip=8.8.8.8 anddest_ip=8.8.8.9) and a part of them are even identical.

note when I’m trying to test it I found that I have on average 9 events in 5 min but when I’m using the real-time search I get almost 130 on average.

Labels (3)

srijondas
Explorer

I am facing a similar problem.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Manage Edge Processor pipelines end to end with APIs

This guide uses the Data Management APIs to create an Edge Processor, onboard one instance, deploy a pipeline ...

Splunk Brings Proof of Control to Digital Asset Transfers

Digital assets now sit inside the core of financial services. HSBC launched its Tokenized Deposit Service in ...

Manage Ingest Processor pipelines end to end with APIs

This guide uses the Data Management APIs to create an Ingest Processor pipeline, deploy it to the ...