Getting empty results when running search with particular fields using Java SDK

[john_dem8]

Does anybody know why while I am able to get results when running query with any field in Splunk, I am  getting empty result when trying to run the same query for particular fields with Java SDK? Does that mean some fields are special somehow?

Sample query: search field1=value1

Java code:

JobArgs jobArgs=new JobArgs();
jobArgs.setEarliest("-1m@m");

String query="search field1=value1";
Job job=splunkService.getJobs().create(query, jobArgs);
while(!job.isDone()) {
    Thread.sleep(500);
}

JobResultsArgs resultsArgs=new JobResultsArgs();
resultsArgs.setOutPutMode(JSON);
job.getResults(resultsArgs);

[PickleRick]

Do you use the same user for searching via webui as with rest?

[john_dem8]

@PickleRick Yeah, I use the same username.

[PickleRick]

That is strange then. Different users could mean permission issues and problem with accessing field extractions definitions.

What I can suggest for troubleshooting, capture the guid, uid or whatever it's called of the rest-initiated job and compare the job details and job log with a web ui launched one.

[john_dem8]

@PickleRick I just tried to run same query with SDK and in Splunk and I didn't get results with SDK. Here is the response:

{"preview":false, "post_process_count":0, "mesages":[], "results":[], "init_offset":0}

I also noticed, the field I used in my query is not a default field in Splunk, it's under "Interesting fields" category.

I just created a job with SDK, copied the SID value and pasted it in Splunk URL:

 https://base_url.com/?sid=... 

but that was redirected to another SID and it displayed results in Splunk.

@PickleRick Is that what you meant to do for troubleshooting?

[jamie00171]

Hi @john_dem8 

Some other things that might be useful for troubleshooting:

You could search the _audit index to view the result_count for the search, if you have the search ID you could do:

index=_audit action=search info=completed search_id=<search ID goes here>

You could also use this to confirm the search is being executed as the same user as the UI search.

If you want to see the results of a search you can run:

| loadjob <search Id> 

You can also use the job inspector for a search by going to the following URL:

 https://base_url.com/en-US/manager/search/job_inspector?sid=<search ID>

There may be an error that shows up here.

Thanks, 

Jamie

[john_dem8]

@jamie00171 I tried to search _audit index but didn't get any results for SID I used with SDK or for one I used directly in Splunk.

I ran this: 

| loadjob <search Id>

and was not able to see any data for SID using SDK.

I also tried this: 

 https://base_url.com/en-US/manager/search/job_inspector?sid=<search ID>

and it said: "This search has completed in 0.98 seconds but did not match any events. The terms specified in the highlighted portion of the search: <my query here>...." I ran that query in Splunk and it returned some events.

So I am still not sure what can be wrong with SDK.

[PickleRick]

Are you searching over the same period from the webui?

Your rest call seems to be only searching over last minute or so.

[PickleRick]

That's what I was talking about 🙂

@john_dem8if you take the search ID you get from your REST call and after loading the job in UI you get results, it most probably means you're not waiting long enough for the results.