Getting Data In

Inconsistency between search results between Splunk UI and Rest API & REST API itself

siddharth1479
Path Finder

Hi Community,
I'm trying to extract search results using REST API and I'm facing the following problem.
1. I'm using the curl command: curl --location --request POST 'https://XXXXXX/services/search/jobs/export' \ --data-urlencode 'search=search index=uam user="abcd" event=auth earliest="01/16/2020:00:00:00" latest=now() | eval access_time=strftime(_time, "%Y-%d-%m %H:%M:%S") | stats count by access_time user status | fields - count' \ --data-urlencode 'output_mode=json'. The results I get are different every time I fire up the API. Sometime there are 10 results, sometimes 20, 15, and so on; i.e. inconsistent.

  1. When I use the same searchquery in the Splunk UI, I get the reults, which are different from the results i get from the API call, which is desired. Search result I use is: index=uam user="abcd" event=auth earliest="01/16/2020:00:00:00" latest=now() | eval access_time=strftime(_time, "%Y-%d-%m %H:%M:%S") | stats count by access_time user status | fields - count.

Along with that, when i get back the json results, there is a field "preview" which have values "true" or "false", I assume that the results with preview: true might be shown in the Splunk UI.

Result from the API call:
`{
"preview": true,
"offset": 9,
"result": {
"access_time": "2020-15-01 18:06:21",
"user": "adcgwjv_ahubt_ext1",
"status": "success"
}
}

{
"preview": false,
"offset": 9,
"result": {
"access_time": "2020-15-01 18:06:21",
"user": "adcgwjv_ahubt_ext1",
"status": "success"
}
}
`

Even after using | dedup access_time, i get the repeated results on API call, but works fine in the Splunk UI.

Please help.

Thanks,
Sid

1 Solution

jkat54
SplunkTrust
SplunkTrust

Bear in mind that when you dispatch a search via UI you are having RBAC applied as well as KOs from the App Context you are in.

When you dispatch a search via REST, you might not be executing the search in the correct name space (servicesNS/user/app/) or you may be executing as a different role.

Please see docs about servicesNS endpoints and try using the same app & user context with your curl command.

View solution in original post

jkat54
SplunkTrust
SplunkTrust

Bear in mind that when you dispatch a search via UI you are having RBAC applied as well as KOs from the App Context you are in.

When you dispatch a search via REST, you might not be executing the search in the correct name space (servicesNS/user/app/) or you may be executing as a different role.

Please see docs about servicesNS endpoints and try using the same app & user context with your curl command.

siddharth1479
Path Finder

Can you elaborate more? I'm fairly new to Splunk and learning to operate. Is there any link i can refer? Also in the above given curl command, can you point out that part of it are you talking about?

Thanks for the help @jkat54.

jkat54
SplunkTrust
SplunkTrust

https://docs.splunk.com/Documentation/Splunk/8.0.1/RESTTUT/RESTsearches

Follow the above and your problem should go away.

You're currently getting results from "services/search/jobs/export" and you should be using something like "services/search/jobs/SID/results" instead

siddharth1479
Path Finder

Yes, now I got to know what you're talking about and what caused the duplication. Thanks for the insights.

So now that you told that, we need to look for "SID" to obtain the results, what i did is created a chained API call which takes sid from one POST call "/services/search/jobs" which gives SID and the using this SID, in to GET call "/services/search/{{sid}}/results".

But again, my question is; Is there anyway which can make this single call or simpler logic for this results generation?

Thanks,
Sid

jkat54
SplunkTrust
SplunkTrust

You can make the search a scheduled report in splunk and then call the latest results of the scheduled report each time instead.

This would have 2 benefits. It would give fast results with less calls to the API, and it can help reduce
the load your code would place on the system. Since you'd always get the results of the last run, you wouldn't be running new searches all the time with your code.

siddharth1479
Path Finder

That's a great idea. You will get almost a real=time solution and also make it more proficient.

Thanks for all the help.

jkat54
SplunkTrust
SplunkTrust

Anytime!!!

0 Karma

jkat54
SplunkTrust
SplunkTrust

This is what you're currently doing:

search/jobs/export
https://:/services/search/jobs/export

"Stream search results as they become available."

Which is why the results keep changing / "duplicating".

efavreau
Motivator

Anytime I use latest=now() I expect differing results. How can the UI results be different all together and desired? Feels like some information is missing about how the data is different. For the sake of comparing apples to apples, fix your time frames and do not make them relative.

###

If this reply helps you, an upvote would be appreciated.
0 Karma

siddharth1479
Path Finder

Hi, Thanks for the reply. Using latest=new() or latest="01/17/2020:11:00:00", the results are inconsistent in both the cases.

And for the missing information, I've edited the post with some more insights.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...