Splunk Dev

How to output raw results from the Splunk Python SDK when using pagination?

reswob4
Builder

So I was having the same problem as here:

https://answers.splunk.com/answers/99633/only-100-results-return-with-python-api-query.html

and I used this: http://dev.splunk.com/view/SP-CAAAEE5#paginating

and this: http://dev.splunk.com/view/python-sdk/SP-CAAAER5

To create a script that now returns ALL my results, BUT I need to write the raw results to a file.

I WAS using the following syntax:

<snip>
kwargs_normalsearch = {"exec_mode":"normal"}
job = service.jobs.create(searchquery, **kwargs_normalsearch)

<check for job to be done>

content = str(job.results(output_mode="raw"))
file.write(content)
<snip>

But I have not figured out how to combine the output_mode="raw" into the pagination script. I have not yet found anything in the above documentation or via Google showing the correct syntax, so I'm asking here.

I basically copied what was on one of the pages listed above:

kwargs_blockingsearch = {"exec_mode":"blocking"}

print "Search results:\n"
resultCount = job["resultCount"]  # Number of results this job returned
offset = 0;                       # Start at result 0
count = 10;                       # Get sets of 10 results at a time

while (offset < int(resultCount)):
    kwargs_paginate = {"count": count, "offset": offset}

    # Get the search results and display them
    blocksearch_results = job.results(**kwargs_paginate)

    for result in results.ResultsReader(blocksearch_results):
        print result

    # Increase the offset to get the next set of results
    offset += count

And I've tried:

blocksearch_results = job.results(**kwargs_paginate, output_mode="raw")

and

blocksearch_results = str(job.results(**kwargs_paginate, output_mode="raw"))

and

kwargs_paginate = {'output_mode": "raw", "count": count, "offset": offset}

all fail due to invalid syntax.

Does anyone have any suggestions?

1 Solution

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

View solution in original post

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...