Splunk Dev

How to output raw results from the Splunk Python SDK when using pagination?

reswob4
Builder

So I was having the same problem as here:

https://answers.splunk.com/answers/99633/only-100-results-return-with-python-api-query.html

and I used this: http://dev.splunk.com/view/SP-CAAAEE5#paginating

and this: http://dev.splunk.com/view/python-sdk/SP-CAAAER5

To create a script that now returns ALL my results, BUT I need to write the raw results to a file.

I WAS using the following syntax:

<snip>
kwargs_normalsearch = {"exec_mode":"normal"}
job = service.jobs.create(searchquery, **kwargs_normalsearch)

<check for job to be done>

content = str(job.results(output_mode="raw"))
file.write(content)
<snip>

But I have not figured out how to combine the output_mode="raw" into the pagination script. I have not yet found anything in the above documentation or via Google showing the correct syntax, so I'm asking here.

I basically copied what was on one of the pages listed above:

kwargs_blockingsearch = {"exec_mode":"blocking"}

print "Search results:\n"
resultCount = job["resultCount"]  # Number of results this job returned
offset = 0;                       # Start at result 0
count = 10;                       # Get sets of 10 results at a time

while (offset < int(resultCount)):
    kwargs_paginate = {"count": count, "offset": offset}

    # Get the search results and display them
    blocksearch_results = job.results(**kwargs_paginate)

    for result in results.ResultsReader(blocksearch_results):
        print result

    # Increase the offset to get the next set of results
    offset += count

And I've tried:

blocksearch_results = job.results(**kwargs_paginate, output_mode="raw")

and

blocksearch_results = str(job.results(**kwargs_paginate, output_mode="raw"))

and

kwargs_paginate = {'output_mode": "raw", "count": count, "offset": offset}

all fail due to invalid syntax.

Does anyone have any suggestions?

1 Solution

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

View solution in original post

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...