Real time search with python sdk and | stats

maurelio79 — Sun, 26 Jan 2014 22:08:13 GMT

Hi guys, i think i'm missing something.
I'm try to make a real time search with python sdk; after connection if i run this search:

search = "search index=main sourcetype=access_combined method=GET"

there is no problem: the shell quite soon start to output something like

OrderedDict([('_confstr', 'source::/home/maurelio/Web/heatmiser/log/access.log|host::DPL101|access_combined'), ('_indextime', '1390773156'), ('_kv', '1'), ('_raw', '79.10.253.90 - - [26/Jan/2014:22:52:32] "GET /jqwidgets/jqxscrollbar.js HTTP/1.1" 304 - "<link_removed>" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"'), ('_serial', '11'), ('_sourcetype', 'access_combined'), ('_time', '2014-01-26 22:52:32.000 CET'), ('host', 'DPL101'), ('index', 'main'), ('linecount', '1'), ('method', 'GET'), ('source', '/home/maurelio/Web/heatmiser/log/access.log'), ('sourcetype', 'access_combined'), ('splunk_server', 'DPL101')])
None
OrderedDict([('_confstr', 'source::/home/maurelio/Web/heatmiser/log/access.log|host::DPL101|access_combined'), ('_indextime', '1390773156'), ('_kv', '1'), ('_raw', '79.10.253.90 - - [26/Jan/2014:22:52:32] "GET /jqwidgets/jqxchart.js HTTP/1.1" 304 - "<link_removed>" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"'), ('_serial', '12'), ('_sourcetype', 'access_combined'), ('_time', '2014-01-26 22:52:32.000 CET'), ('host', 'DPL101'), ('index', 'main'), ('linecount', '1'), ('method', 'GET'), ('source', '/home/maurelio/Web/heatmiser/log/access.log'), ('sourcetype', 'access_combined'), ('splunk_server', 'DPL101')])
None

But if i change my search in:

search = "search index=main sourcetype=access_combined method=GET | stats count by status"

i get no output in the shell; the script stay there without "output" nothing.
Of course the seme search works in splunk web.

What i'm missing?
Thanks

UPDATE
Ok i answer to my self: simply it takes many minutes to start (about 3/4 minutes) but after that it starts, by the way i'd like to know why this big difference.

UPDATE WITH CODE

search = "search index=main sourcetype=access_combined method=GET | stats count by status"
    DATA = []

    try:
        result = service.get(
            "search/jobs/export",
            search=search,
            earliest_time="rt", 
            latest_time="rt", 
            search_mode="realtime")

        for result in ResultsReader(result.body):
            if result is not None:
              # DATA.append([result["status"], result["count"]])
                print(result)


    except KeyboardInterrupt:
            print "\nInterrupted."

Re: Real time search with python sdk and | stats

Damien_Dallimor — Sun, 26 Jan 2014 23:28:58 GMT

Can I see your python code ?

Re: Real time search with python sdk and | stats

maurelio79 — Sun, 26 Jan 2014 23:35:38 GMT

Of course! Updated my post.

Re: Real time search with python sdk and | stats

s2upin — Sat, 08 Apr 2017 14:38:20 GMT

thanks you !

Re: Real time search with python sdk and | stats

jkat54 — Tue, 29 Sep 2020 13:37:38 GMT

The stats command is waiting until the map reduce queue fills before returning results...

Try tweaking the rt_queue_size or reduce_freq search options that are mentioned here:

http://dev.splunk.com/view/python-sdk/SP-CAAAEE5

topic Real time search with python sdk and | stats in Splunk Search

Real time search with python sdk and | stats

Re: Real time search with python sdk and | stats

Re: Real time search with python sdk and | stats

Re: Real time search with python sdk and | stats

Re: Real time search with python sdk and | stats