Developing for Splunk Enterprise

Splunk API taking long time through java splunk sdk

bubby248
New Member

Hi
I am using saved searches to retrieve the results from Splunk.However after indexing, the data in Splunk web UI is faster, but the api response from java sdk is still slow.
Can you guys help me out regarding this.

Thanks
Harsha

Tags (2)
0 Karma

bubby248
New Member

No my saved searches are not scheduled.Is there anyway I can get faster response without scheduling saved searches.

0 Karma

kicksammy
Explorer

even i am facing this issue. please help. i see the search getting completed on splunk UI but not through java client.

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

The SDK doesn't add any extra overhead and the response time should be very similar to the Splunk UI. You mentioned that you are using saved searches to pull data from Splunk. Is the saved search scheduled? If so, you should use the history function on the saved search object to retrieve the last run instance. That should be speed things up hopefully.

0 Karma

andras_kerekes
Explorer

How big is the index you search?

I have noticed that searching an index of size 1m events if I did not limit the results with | head 50000 (could be other value es well, I used this to be in sync with the REST API result limit in limits.conf) Splunk will search through all 1m event and then return 50000 (as it is configured in limits.conf)

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

Shouldn't be any difference.

0 Karma

bubby248
New Member

Below is the code am using from java
Job job =null;
Map inputArgs = new HashMap();
inputArgs.put("ttl",30);
job = service.getJobs().create("|savedsearch mysavedsearch earliest=-1440m@m latest=now span=1hr",inputArgs);
while (!job.isDone())
{try Thread.sleep(2000);}catch (InterruptedException e) {}
job.refresh();

} Map outputArgs = new HashMap(); outputArgs.put("count",0);//To get more than 100 results
outputArgs.put("output_mode","json"); InputStream stream = job.getResults(outputArgs);

0 Karma

bubby248
New Member

Thanks Neeraj.

My main concern is API is taking more time(30 to 40 sec) some times, and sometimes it is very fast(5 to 6 sec).

In Splunk UI it is very fast,within 2 sec am getting the results.
My Java Application will be hitting by multiple users.Is it creating any slowness.

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

No, this will also work. You are doing the same thing, creating a job from a saved search.

0 Karma

bubby248
New Member

Service.getJobs().create("mysavedsearch"',inputargs)

Here am just retreiving the savedsearch right.Am I creating any new saved searches on the server.I just need to retreive it

0 Karma

bubby248
New Member

Thanks Neeraj

0 Karma

Neeraj_Luthra
Splunk Employee
Splunk Employee

This should work. Another way to approach this would be to create/save the search on the server and then dispatch it as follows to get a handle on the Job. With this approach, you can decide to schedule it in the fure and then use the history method for faster returns.

SavedSearch ss = service.getSavedSearches().get("my_saved_search");
Job job = ss.dispatch();
while (!job.isReady()) {
    Thread.sleep(1000);
}
0 Karma

bubby248
New Member

Hi Neeraj

In my java code am using in below format
Service.getJobs().create("mysavedsearch"',inputargs)
From above am retreiving the job details .Is this approach is a good one.

0 Karma

bubby248
New Member

No my saved searches are not scheduled.Is there anyway I can get faster response without scheduling saved searches.
(20 secs ago)bubby248

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.