topic Re: Splunk API taking long time through java splunk sdk in Splunk Dev

Splunk API taking long time through java splunk sdk

bubby248 — Mon, 04 Mar 2013 23:03:52 GMT

Hi
I am using saved searches to retrieve the results from Splunk.However after indexing, the data in Splunk web UI is faster, but the api response from java sdk is still slow.
Can you guys help me out regarding this.

Thanks
Harsha

Re: Splunk API taking long time through java splunk sdk

Neeraj_Luthra — Mon, 04 Mar 2013 23:26:56 GMT

The SDK doesn't add any extra overhead and the response time should be very similar to the Splunk UI. You mentioned that you are using saved searches to pull data from Splunk. Is the saved search scheduled? If so, you should use the history function on the saved search object to retrieve the last run instance. That should be speed things up hopefully.

Re: Splunk API taking long time through java splunk sdk

bubby248 — Tue, 05 Mar 2013 00:33:44 GMT

No my saved searches are not scheduled.Is there anyway I can get faster response without scheduling saved searches.

Re: Splunk API taking long time through java splunk sdk

bubby248 — Tue, 05 Mar 2013 00:34:43 GMT

No my saved searches are not scheduled.Is there anyway I can get faster response without scheduling saved searches.
(20 secs ago)bubby248

Re: Splunk API taking long time through java splunk sdk

bubby248 — Tue, 05 Mar 2013 01:02:00 GMT

Hi Neeraj

In my java code am using in below format
Service.getJobs().create("mysavedsearch"',inputargs)
From above am retreiving the job details .Is this approach is a good one.

Re: Splunk API taking long time through java splunk sdk

Neeraj_Luthra — Wed, 06 Mar 2013 09:47:57 GMT

This should work. Another way to approach this would be to create/save the search on the server and then dispatch it as follows to get a handle on the Job. With this approach, you can decide to schedule it in the fure and then use the history method for faster returns.

SavedSearch ss = service.getSavedSearches().get("my_saved_search");
Job job = ss.dispatch();
while (!job.isReady()) {
    Thread.sleep(1000);
}

Re: Splunk API taking long time through java splunk sdk

bubby248 — Wed, 06 Mar 2013 22:10:12 GMT

Thanks Neeraj

Re: Splunk API taking long time through java splunk sdk

bubby248 — Wed, 06 Mar 2013 23:03:18 GMT

Service.getJobs().create("mysavedsearch"',inputargs)

Here am just retreiving the savedsearch right.Am I creating any new saved searches on the server.I just need to retreive it

Re: Splunk API taking long time through java splunk sdk

Neeraj_Luthra — Thu, 07 Mar 2013 10:38:54 GMT

No, this will also work. You are doing the same thing, creating a job from a saved search.

Re: Splunk API taking long time through java splunk sdk

bubby248 — Thu, 07 Mar 2013 17:08:00 GMT

Thanks Neeraj.

My main concern is API is taking more time(30 to 40 sec) some times, and sometimes it is very fast(5 to 6 sec).

In Splunk UI it is very fast,within 2 sec am getting the results.
My Java Application will be hitting by multiple users.Is it creating any slowness.

Re: Splunk API taking long time through java splunk sdk

bubby248 — Thu, 07 Mar 2013 17:11:45 GMT

Below is the code am using from java
Job job =null;
Map inputArgs = new HashMap();
inputArgs.put("ttl",30);
job = service.getJobs().create("|savedsearch mysavedsearch earliest=-1440m@m latest=now span=1hr",inputArgs);
while (!job.isDone())
{try Thread.sleep(2000);}catch (InterruptedException e) {}
job.refresh();

} Map outputArgs = new HashMap(); outputArgs.put("count",0);//To get more than 100 results
outputArgs.put("output_mode","json"); InputStream stream = job.getResults(outputArgs);

Re: Splunk API taking long time through java splunk sdk

Neeraj_Luthra — Mon, 11 Mar 2013 23:19:28 GMT

Shouldn't be any difference.

Re: Splunk API taking long time through java splunk sdk

andras_kerekes — Thu, 27 Jun 2013 20:49:03 GMT

How big is the index you search?

I have noticed that searching an index of size 1m events if I did not limit the results with | head 50000 (could be other value es well, I used this to be in sync with the REST API result limit in limits.conf) Splunk will search through all 1m event and then return 50000 (as it is configured in limits.conf)

Re: Splunk API taking long time through java splunk sdk

kicksammy — Wed, 26 Nov 2014 01:44:38 GMT

even i am facing this issue. please help. i see the search getting completed on splunk UI but not through java client.