Does using base searches increase the performance of the dashboards?

[buttsurfer]

So from my research it looks like Base Searches increase the performance of the dashboards. A dashboard with several views loads faster if the query of each view is using a pre-existing base search. However my friend is convinced that that's not the case and using Base Searches does the opposite - it prolongs the loading time of the dashboard. Has anyone else had such experience?

[PickleRick · Accepted solution]

The only answer to such generally stated question must be "it depends".

There are some cases when using base searches will be detrimental to the performance of the dashboard - typically if you return a big set of "raw" (non aggregated) events which you further process in your searches.

But if used properly, they do improve performance - you get one "heavy" search job which queries the indexes and calculates detailed statistics and then use lightwight post-process searches to aggregate or filter. For example, you do a base search of logins counting them over users and workstations. Then you do one table with stats overs users and another with stats over workstations. Or you filter the results into several separate graphs by business units. You do the hard work (going through the index) once, and reuse the results several times.

[jamie00171]

My experience is very much the same as this. I find base searches which are too generic and return a large number of non aggregate events perform very poorly. In addition, they also use a large amount of disk space which contributes to the users disk quota (until the search artifacts are removed). This can in turn lead to a lot of queuing for the users search jobs so it's worth noting how much disk space base searches use via the job inspector or the _introspection logs when developing the dashboard.

In general I've found the best approach is to use base searches for summary panels at the top of dashboards which use aggregated data (e.g. single value visualizations) and avoid base searches when you want to display more non aggregated events (e.g. in a table). 

One other thing to note here is that, if the dashboard is very heavily used and you can have base searches to reduce the number of searches that run when the dashboard loads (or alternatively have panels behind check boxes etc.) then you could reduce your concurrent search load on your Splunk servers. This could be important in large distributed environments with a large number of concurrent users. In this case you might be happy to have the dashboard take a few extra seconds to load if it results in less searches being executed.  

[richgalloway]

Yes, using base searches improves the performance of a dashboard.  Of course, that's a generalization and there may be specific cases where a base search degraded performance, but I'd wager it was done wrong in that case.

Your friend should show his before and after metrics.