- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Community,
I'm trying to extract search results using REST API and I'm facing the following problem.
1. I'm using the curl command: curl --location --request POST 'https://XXXXXX/services/search/jobs/export' \ --data-urlencode 'search=search index=uam user="abcd" event=auth earliest="01/16/2020:00:00:00" latest=now() | eval access_time=strftime(_time, "%Y-%d-%m %H:%M:%S") | stats count by access_time user status | fields - count' \ --data-urlencode 'output_mode=json'
. The results I get are different every time I fire up the API. Sometime there are 10 results, sometimes 20, 15, and so on; i.e. inconsistent.
- When I use the same searchquery in the Splunk UI, I get the reults, which are different from the results i get from the API call, which is desired. Search result I use is:
index=uam user="abcd" event=auth earliest="01/16/2020:00:00:00" latest=now() | eval access_time=strftime(_time, "%Y-%d-%m %H:%M:%S") | stats count by access_time user status | fields - count
.
Along with that, when i get back the json results, there is a field "preview" which have values "true" or "false", I assume that the results with preview: true might be shown in the Splunk UI.
Result from the API call:
`{
"preview": true,
"offset": 9,
"result": {
"access_time": "2020-15-01 18:06:21",
"user": "adcgwjv_ahubt_ext1",
"status": "success"
}
}
{
"preview": false,
"offset": 9,
"result": {
"access_time": "2020-15-01 18:06:21",
"user": "adcgwjv_ahubt_ext1",
"status": "success"
}
}
`
Even after using | dedup access_time
, i get the repeated results on API call, but works fine in the Splunk UI.
Please help.
Thanks,
Sid
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Bear in mind that when you dispatch a search via UI you are having RBAC applied as well as KOs from the App Context you are in.
When you dispatch a search via REST, you might not be executing the search in the correct name space (servicesNS/user/app/) or you may be executing as a different role.
Please see docs about servicesNS endpoints and try using the same app & user context with your curl command.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Bear in mind that when you dispatch a search via UI you are having RBAC applied as well as KOs from the App Context you are in.
When you dispatch a search via REST, you might not be executing the search in the correct name space (servicesNS/user/app/) or you may be executing as a different role.
Please see docs about servicesNS endpoints and try using the same app & user context with your curl command.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you elaborate more? I'm fairly new to Splunk and learning to operate. Is there any link i can refer? Also in the above given curl command, can you point out that part of it are you talking about?
Thanks for the help @jkat54.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
https://docs.splunk.com/Documentation/Splunk/8.0.1/RESTTUT/RESTsearches
Follow the above and your problem should go away.
You're currently getting results from "services/search/jobs/export" and you should be using something like "services/search/jobs/SID/results" instead
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, now I got to know what you're talking about and what caused the duplication. Thanks for the insights.
So now that you told that, we need to look for "SID" to obtain the results, what i did is created a chained API call which takes sid from one POST call "/services/search/jobs" which gives SID and the using this SID, in to GET call "/services/search/{{sid}}/results".
But again, my question is; Is there anyway which can make this single call or simpler logic for this results generation?
Thanks,
Sid
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can make the search a scheduled report in splunk and then call the latest results of the scheduled report each time instead.
This would have 2 benefits. It would give fast results with less calls to the API, and it can help reduce
the load your code would place on the system. Since you'd always get the results of the last run, you wouldn't be running new searches all the time with your code.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That's a great idea. You will get almost a real=time solution and also make it more proficient.
Thanks for all the help.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Anytime!!!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is what you're currently doing:
search/jobs/export
https://:/services/search/jobs/export
"Stream search results as they become available."
Which is why the results keep changing / "duplicating".
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Anytime I use latest=now()
I expect differing results. How can the UI results be different all together and desired? Feels like some information is missing about how the data is different. For the sake of comparing apples to apples, fix your time frames and do not make them relative.
If this reply helps you, an upvote would be appreciated.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, Thanks for the reply. Using latest=new()
or latest="01/17/2020:11:00:00"
, the results are inconsistent in both the cases.
And for the missing information, I've edited the post with some more insights.