Splunk Search

Real time search with python sdk and | stats

maurelio79
Communicator

Hi guys, i think i'm missing something.
I'm try to make a real time search with python sdk; after connection if i run this search:

search = "search index=main sourcetype=access_combined method=GET"

there is no problem: the shell quite soon start to output something like

OrderedDict([('_confstr', 'source::/home/maurelio/Web/heatmiser/log/access.log|host::DPL101|access_combined'), ('_indextime', '1390773156'), ('_kv', '1'), ('_raw', '79.10.253.90 - - [26/Jan/2014:22:52:32] "GET /jqwidgets/jqxscrollbar.js HTTP/1.1" 304 - "<link_removed>" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"'), ('_serial', '11'), ('_sourcetype', 'access_combined'), ('_time', '2014-01-26 22:52:32.000 CET'), ('host', 'DPL101'), ('index', 'main'), ('linecount', '1'), ('method', 'GET'), ('source', '/home/maurelio/Web/heatmiser/log/access.log'), ('sourcetype', 'access_combined'), ('splunk_server', 'DPL101')])
None
OrderedDict([('_confstr', 'source::/home/maurelio/Web/heatmiser/log/access.log|host::DPL101|access_combined'), ('_indextime', '1390773156'), ('_kv', '1'), ('_raw', '79.10.253.90 - - [26/Jan/2014:22:52:32] "GET /jqwidgets/jqxchart.js HTTP/1.1" 304 - "<link_removed>" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"'), ('_serial', '12'), ('_sourcetype', 'access_combined'), ('_time', '2014-01-26 22:52:32.000 CET'), ('host', 'DPL101'), ('index', 'main'), ('linecount', '1'), ('method', 'GET'), ('source', '/home/maurelio/Web/heatmiser/log/access.log'), ('sourcetype', 'access_combined'), ('splunk_server', 'DPL101')])
None

But if i change my search in:

search = "search index=main sourcetype=access_combined method=GET | stats count by status"

i get no output in the shell; the script stay there without "output" nothing.
Of course the seme search works in splunk web.

What i'm missing?
Thanks

UPDATE
Ok i answer to my self: simply it takes many minutes to start (about 3/4 minutes) but after that it starts, by the way i'd like to know why this big difference.

UPDATE WITH CODE

search = "search index=main sourcetype=access_combined method=GET | stats count by status"
    DATA = []

    try:
        result = service.get(
            "search/jobs/export",
            search=search,
            earliest_time="rt", 
            latest_time="rt", 
            search_mode="realtime")

        for result in ResultsReader(result.body):
            if result is not None:
              # DATA.append([result["status"], result["count"]])
                print(result)


    except KeyboardInterrupt:
            print "\nInterrupted."

jkat54
SplunkTrust
SplunkTrust

The stats command is waiting until the map reduce queue fills before returning results...

Try tweaking the rt_queue_size or reduce_freq search options that are mentioned here:

http://dev.splunk.com/view/python-sdk/SP-CAAAEE5

0 Karma

maurelio79
Communicator

Of course! Updated my post.

0 Karma

s2upin
Explorer

thanks you !

0 Karma

Damien_Dallimor
Ultra Champion

Can I see your python code ?

0 Karma
Get Updates on the Splunk Community!

This Week's Community Digest - Splunk Community Happenings [9.26.22]

Get the latest news and updates from the Splunk Community here! Upcoming User Group Events! &#x1f44f; Check ...

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...