Splunk Dev

How to output raw results from the Splunk Python SDK when using pagination?

reswob4
Builder

So I was having the same problem as here:

https://answers.splunk.com/answers/99633/only-100-results-return-with-python-api-query.html

and I used this: http://dev.splunk.com/view/SP-CAAAEE5#paginating

and this: http://dev.splunk.com/view/python-sdk/SP-CAAAER5

To create a script that now returns ALL my results, BUT I need to write the raw results to a file.

I WAS using the following syntax:

<snip>
kwargs_normalsearch = {"exec_mode":"normal"}
job = service.jobs.create(searchquery, **kwargs_normalsearch)

<check for job to be done>

content = str(job.results(output_mode="raw"))
file.write(content)
<snip>

But I have not figured out how to combine the output_mode="raw" into the pagination script. I have not yet found anything in the above documentation or via Google showing the correct syntax, so I'm asking here.

I basically copied what was on one of the pages listed above:

kwargs_blockingsearch = {"exec_mode":"blocking"}

print "Search results:\n"
resultCount = job["resultCount"]  # Number of results this job returned
offset = 0;                       # Start at result 0
count = 10;                       # Get sets of 10 results at a time

while (offset < int(resultCount)):
    kwargs_paginate = {"count": count, "offset": offset}

    # Get the search results and display them
    blocksearch_results = job.results(**kwargs_paginate)

    for result in results.ResultsReader(blocksearch_results):
        print result

    # Increase the offset to get the next set of results
    offset += count

And I've tried:

blocksearch_results = job.results(**kwargs_paginate, output_mode="raw")

and

blocksearch_results = str(job.results(**kwargs_paginate, output_mode="raw"))

and

kwargs_paginate = {'output_mode": "raw", "count": count, "offset": offset}

all fail due to invalid syntax.

Does anyone have any suggestions?

1 Solution

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

View solution in original post

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...