topic Re: REST API for pre-sorted data in Splunk Search

REST API for pre-sorted data

abarkerSendGrid — Tue, 02 Oct 2012 23:26:13 GMT

Hi Splunk Pro's,

I'm looking for a way to grab processed, sorted data via a REST API call. For instance when logged into the Splunk search dashboard, I can get a sorted list of data based on an event that I'm able to specify:

"event=spamreport | top limit=50 userid | table userid, count"

which processes the raw log data and returns an organized list of userid's sorted by their total spam report count.

Is there a way to get this pre-processed, sorted data via a REST API call? Using the same search string above, I've tried both synchronously returning results with /search/search/jobs/export as well as asynchronously returning results with /search/search/jobs to create a job and /search/search/jobs/JOBID/results to retrieve the data once the job is complete. However, both approaches only give me the raw data, not the sorted list that I can see from the web search dashboard. I've tried playing with the output_mode, but no option that I'm aware of produces the desired result.

Are there any options other than a REST API call? I'm looking for something that is easily doable from the command line or a script and that doesn't require running the search from the same server that contains the log data.

Thanks very much!

Re: REST API for pre-sorted data

Damien_Dallimor — Wed, 03 Oct 2012 07:13:22 GMT

Have you tried adding the sort command into the search pipeline you are using in the REST call

search event=spamreport | top limit=50 userid | table userid, count | sort - count

Re: REST API for pre-sorted data

abarkerSendGrid — Wed, 03 Oct 2012 14:45:28 GMT

Adding the sort command didn't get me what I'm looking for. Neither for both synchronous and asynchronous searches.

Let me describe more specifically what I'm looking for. Below is the format of output I would like to get from the REST call.

userid count
340930 496
129298 434
34465 128
100891 80

This is possible by running the following search command directly on the server where the data exists.

/opt/splunk/bin/splunk search "search event=spamreport | top limit=50 userid | table userid, count | sort - count" output=rawdata -auth $SPLUNKUSER:$SPLUNKPASS 2>/dev/null

Re: REST API for pre-sorted data

abarkerSendGrid — Wed, 03 Oct 2012 14:48:21 GMT

Is this aggregated output format possible with a REST API call? Is there another way to go about this? The only requirement for me is that the API search doesn't require itself to be run on the same machine as the data.

Re: REST API for pre-sorted data

Neeraj_Luthra — Mon, 28 Sep 2020 12:33:58 GMT

If you specify the "f" parameter in your request, you should be able to get your desired aggregated output. You can do this with both jobs/${search_id}/results as well as jobs/export. Below is a small code snippet for both of these calls.

curl -k -u admin:changeme https://localhost:8089/services/search/jobs/export -d search="search index%3D_internal | top limit%3D10 sourcetype" -d output_mode=csv -d f=sourcetype -d f=count OR curl -k -u admin:changeme https://localhost:8089/services/search/jobs/${search_id]/results --get -d output_mode=csv -d f=sourcetype -d f=count

Also, I am not sure if you are aware of the Splunk SDKs (Java, JavaScript, Python, PHP) but I would encourage accessing the REST APIs through them as they make it easier for you to manage and work with Splunk objects and you can stay in the comfort zone of your preferred language.

If you need help or code sample to implement the commands above through any of the SDKs, feel free to contact me.

Hope this helps.