Splunk Search

Real time search with python sdk and | stats

maurelio79
Communicator

Hi guys, i think i'm missing something.
I'm try to make a real time search with python sdk; after connection if i run this search:

search = "search index=main sourcetype=access_combined method=GET"

there is no problem: the shell quite soon start to output something like

OrderedDict([('_confstr', 'source::/home/maurelio/Web/heatmiser/log/access.log|host::DPL101|access_combined'), ('_indextime', '1390773156'), ('_kv', '1'), ('_raw', '79.10.253.90 - - [26/Jan/2014:22:52:32] "GET /jqwidgets/jqxscrollbar.js HTTP/1.1" 304 - "<link_removed>" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"'), ('_serial', '11'), ('_sourcetype', 'access_combined'), ('_time', '2014-01-26 22:52:32.000 CET'), ('host', 'DPL101'), ('index', 'main'), ('linecount', '1'), ('method', 'GET'), ('source', '/home/maurelio/Web/heatmiser/log/access.log'), ('sourcetype', 'access_combined'), ('splunk_server', 'DPL101')])
None
OrderedDict([('_confstr', 'source::/home/maurelio/Web/heatmiser/log/access.log|host::DPL101|access_combined'), ('_indextime', '1390773156'), ('_kv', '1'), ('_raw', '79.10.253.90 - - [26/Jan/2014:22:52:32] "GET /jqwidgets/jqxchart.js HTTP/1.1" 304 - "<link_removed>" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"'), ('_serial', '12'), ('_sourcetype', 'access_combined'), ('_time', '2014-01-26 22:52:32.000 CET'), ('host', 'DPL101'), ('index', 'main'), ('linecount', '1'), ('method', 'GET'), ('source', '/home/maurelio/Web/heatmiser/log/access.log'), ('sourcetype', 'access_combined'), ('splunk_server', 'DPL101')])
None

But if i change my search in:

search = "search index=main sourcetype=access_combined method=GET | stats count by status"

i get no output in the shell; the script stay there without "output" nothing.
Of course the seme search works in splunk web.

What i'm missing?
Thanks

UPDATE
Ok i answer to my self: simply it takes many minutes to start (about 3/4 minutes) but after that it starts, by the way i'd like to know why this big difference.

UPDATE WITH CODE

search = "search index=main sourcetype=access_combined method=GET | stats count by status"
    DATA = []

    try:
        result = service.get(
            "search/jobs/export",
            search=search,
            earliest_time="rt", 
            latest_time="rt", 
            search_mode="realtime")

        for result in ResultsReader(result.body):
            if result is not None:
              # DATA.append([result["status"], result["count"]])
                print(result)


    except KeyboardInterrupt:
            print "\nInterrupted."

jkat54
SplunkTrust
SplunkTrust

The stats command is waiting until the map reduce queue fills before returning results...

Try tweaking the rt_queue_size or reduce_freq search options that are mentioned here:

http://dev.splunk.com/view/python-sdk/SP-CAAAEE5

0 Karma

maurelio79
Communicator

Of course! Updated my post.

0 Karma

s2upin
Explorer

thanks you !

0 Karma

Damien_Dallimor
Ultra Champion

Can I see your python code ?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...