topic Re: Global search versus loading search by SID - what are the tradeoffs/impact on the search head? in Deployment Architecture

Global search versus loading search by SID - what are the tradeoffs/impact on the search head?

rjthibod — Mon, 01 Feb 2016 12:15:57 GMT

Splunk Enterprise 6.3.x has added lots of features that greatly extend the Simple XML framework. One capability enables saving the job SID for a completed search (see example XML below). That saved SID can then be accessed elsewhere in the dashboard to load the results from the SID (i.e. using the loadjob command).

There are many ways that this method of accessing search results in a single dashboard is more flexible than using a global search and post-processing. Are there any downsides to using the saved SID approach? Is one more efficient than the other terms of memory, dispatching, etc.?

<search>
  <query>
    <INSERT_SPL_SEARCH>
  </query>
  <earliest>-1h</earliest>
  <latest>now</latest>
  <progress>
    <condition match="'job.resultCount' > 0">
       <set token="search_ds_1_sid">$job.sid$</set>
    </condition>
    <condition>
        <unset token="search_ds_1_sid"/>
     </condition>
  </progress>
</search>

Re: Global search versus loading search by SID - what are the tradeoffs/impact on the search head?

MuS — Mon, 01 Feb 2016 15:33:31 GMT

Hi rjthibod,

using a global search and post-process has limits regarding the results, see docs http://docs.splunk.com/Documentation/Splunk/6.3.2/Viz/Savedsearches#Post-process_searches for more details.

Using the loadjob approach has advantage if you use it in dashboards that are used by may people, this way the schedules saved search runs only once and everyone can use the result (eq. lesser network traffic between SH and IDX and lower performance impact if the dashboard is used by 200 or more people for example).

In the end it all depends on your use case ...

Hope this helps ...

cheers, MuS

Re: Global search versus loading search by SID - what are the tradeoffs/impact on the search head?

rjthibod — Mon, 01 Feb 2016 16:11:36 GMT

Thank you for pointing out the post-process limitations page.

However, I think you misunderstand how I am talking about using loadjob and SIDs. Note, 6.3.X allows one to get the SID of any arbitrary search in a Simple XML panel (see example XML in original post). This not a scheduled saved search in a dashboard, this is the SID of any search in a dashboard regardless of it being saved or not.

Right now, my dashboards have a lot of customizable field selection, so having a saved search is not an option. So, I want to know if I am going to be introducing undesirables in terms of performance/resource utilization if I used the SIDs of my ad-hoc dashboard searches instead of using a global search.

Re: Global search versus loading search by SID - what are the tradeoffs/impact on the search head?

MuS — Mon, 01 Feb 2016 16:25:34 GMT

You're right, got the wrong - sorry. Well to be honest: no one, besides you, can answer this for your use case and environment. Setup two dashboards, one using post process and one using loadjob and to some tests with your data in your setup. You will soon see what is the best setup for you.....

Re: Global search versus loading search by SID - what are the tradeoffs/impact on the search head?

rjthibod — Tue, 02 Feb 2016 13:23:11 GMT

I will try to do that this week and come back with results.