Java sdk Splunk queries performance challenge

[1234testtest]

Hi, I am using JAVA SDK of Splunk and using struts2 as the framework. I have a performance challenge.
Each time the page loads, the queries take too long time to execute (i have 6 of them in each page). So, I implemented Ajax.
But even then, the first time the page loads - takes too long (about 30-40 seconds) (for splunk server connection and then querying the saved searches) before the user sees a response. I am using non-blocking mode of executing which was included in the website (http://dev.splunk.com/view/splunk-java-sdk-how-to/SP-CAAAEKY).

Could you pl suggest a better way of improving the performance.

   Job jobSavedSearch = null;

        // Run the saved search
        try {
            jobSavedSearch = savedSearch.dispatch();
            amJob.job = jobSavedSearch;
        } catch (InterruptedException e1) {
            e1.printStackTrace();
        }

        System.out.println("Waiting for the job to finish...\n");

        // Wait for the job to finish
        while (!jobSavedSearch.isDone()) {
            try {
                Thread.sleep(500);
            } catch (InterruptedException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }

[Neeraj_Luthra · Accepted solution]

The connection to Splunk is most likely not adding any significant delay. My initial guess is that your saved searches are expensive and are taking quite some time to return.

Unless these queries absolutely need to return real-time data, I would recommend scheduling these saved searches on Splunk to run at regular intervals. You can do this from the Splunk UI and pick time intervals like once a minute, hour, day, month etc., whatever suits your business need.

Finally, try the following in your Java code:

SavedSearch mySavedSearch = service.getSavedSearches().get("mySavedSearchName");
Job[] jobs = mySavedSearch.history();
Job myJob = null;
try {
    myJob = (jobs.length > 0) ? jobs[0] : mySavedSearch.dispatch();
} catch (InterruptedException e) {
    e.printStackTrace();
}
System.out.println("Waiting for the job to finish for saved search - " + mySavedSearch.getName() + " ...\n");

while (!myJob.isDone()) {
    try {
        Thread.sleep(500);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}
System.out.println("Job for saved search - " + mySavedSearch.getName() + " finished.\n");

[1234testtest]

Thank you..

[Neeraj_Luthra]

Summary Indexing can definitely help but I suggest that you read through Report Acceleration and Summary Indexing along with their use cases at http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Aboutsummaryindexing.

[1234testtest]

Hi Neeraj,
Thank you for the guidance. Could you kindly suggest does summary indexing help.

[1234testtest]

Thank you. I see that in 5.x version of SPlunk,there is summary indexing, which seemingly is much faster. http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Usesummaryindexing

[Neeraj_Luthra]

That won't help. An index in Splunk is not like one in a typical RDBMS where indexes are created for performance improvement. If you have scheduled your saved searches and are still looking for further improvement, I recommend reading through the Splunk search manual - http://docs.splunk.com/Documentation/Splunk/latest/SearchReference.

[1234testtest]

Does usage of indexes help in improving the performance? Am just evaluating other options as well. Thank you.

[Neeraj_Luthra]

1. The code is set up to dispatch the saved search in the event that the history is empty. That is probably why you are still getting results.
2. After you login into Splunk web, select the appropriate app from the "App" drop down in the top right corner, then click on "Jobs". The URL is something like this - http://$host:$port/en-US/app/$app/job_management. Here you should see jobs for your saved searches, assuming they are scheduled.

[1234testtest]

I have now tried with multiple queries. There are few observations.
1. I dont have scheduled search- but still am getting the results from the savedsearch with the above code. What could be the reason.
2. Could you kindly clarify if there is any splunk web equivalent for the above code - how do I check from where it is getting savedsearch.history.
Kindly help

[1234testtest]

Thank you. It worked!