Dashboard loading is very slow

[Od11]

Hi,

I have a dashboard that uses a base search for all the panels.

when I run the base search outside of the dashboard - it takes like 7 seconds to complete.

but when I open the dashboard - the panels are completely loaded only after 1 minute or more.

The dashboard has many panels and many filters and tokens - could it affect the slow performance? 

how can I improve the performance?

Thanks.

[ITWhisperer]

It may depend on what your dashboard is showing. For example, I have a dashboard which is purely based on events from the last 28 days, not including today, so I schedule a report to be run in the early hours (avoiding 1am to 2am due to daylight-saving clock changes) which I then reference in my base search. The base search still has to load these results but it is much quicker, and once the results are loaded, there are further searches using the base search results to filter and manipulate the results. Overall, the dashboard loads in seconds. The same principle could be applied if you were basing your dashboard on the previous 24 hours for example by running your report every hour.

[Od11]

my issue is not the loading of the query in the base search, it is the dashboard loading.

because the search completes really quick.

[ITWhisperer]

How are you using the base search in your panel searches? Can you share the Simple XML?

[Od11]

I cannot share the whole xml, it is also really huge (like 2500 rows) 

but I can give you a sample from it:

<search>
<query>| inputlookup test1_summary_latest_source.csv|eval orig_source=replace(orig_source,"\\\\","\\\\\\\\")</query>
<done><set token="latest_source">$result.orig_source$</set></done>
</search>

<search id="index_summary_last_base">
<query>
index=test1_summary orig_source="$latest_source$"
| makemv IP delim="--"
| makemv BU delim="--"
| makemv Domain delim="--"
| makemv Regular_LA delim="--"
| makemv A delim="--"
| makemv B delim="--"
| makemv C delim="--"
| makemv D delim="--"

| table A B C D ....(+ another ~50 fields)
</query>
<earliest>-7d@h</earliest>
<latest>now</latest>
</search>

<search id="index_summary_last" base="index_summary_last_base">
<query>
|search BU=$bu$ AND A $a$ AND B $b$ ....(+ another 14 fields/tokens to search)
</query>
</search>

<row>
<panel>
<input type="dropdown" token="bu" searchWhenChanged="true">
<label>BU</label>
<choice value="*">All</choice>
<default>*</default>
<prefix>"</prefix>
<suffix>"</suffix>
<initialValue>*</initialValue>
<fieldForLabel>BU</fieldForLabel>
<fieldForValue>BU</fieldForValue>
<search base="index_summary_last_base">
<query>
| stats count by BU</query>
</search>
</input>

<input type="multiselect" token="a" searchWhenChanged="true">
<label>A</label>
<search base="index_summary_last_base">
<query>|search BU=$bu$
| stats count by A</query>
</search>
<fieldForLabel>A</fieldForLabel>
<fieldForValue>A</fieldForValue>
<choice value="*">All</choice>
<delimiter> ,</delimiter>
<default>*</default>
<prefix>IN (</prefix>
<suffix>)</suffix>
<initialValue>*</initialValue>
<valuePrefix>"</valuePrefix>
<valueSuffix>"</valueSuffix>
</input>

<input type="multiselect" token="b" searchWhenChanged="true">
<label>B</label>
<search base="index_summary_last_base">
<query>|search BU=$bu$ AND A $a$
| stats count by B</query>
</search>
<fieldForLabel>B</fieldForLabel>
<fieldForValue>B</fieldForValue>
<choice value="*">All</choice>
<delimiter> ,</delimiter>
<default>*</default>
<prefix>IN (</prefix>
<suffix>)</suffix>
<initialValue>*</initialValue>
<valuePrefix>"</valuePrefix>
<valueSuffix>"</valueSuffix>
</input>

</panel>
</row>

<row>
<panel>
<title>Last Updated on :</title>
<viz type="status_indicator_app.status_indicator">
<title>Frequency: every 3 days</title>
<search base="index_summary_last_base">
<query>
| stats max(_time) as latest_upload 
| convert ctime("latest_upload") timeformat="%m/%d/%Y %H:%m"</query>
</search>
<option name="drilldown">none</option>
<option name="height">62</option>
<option name="refresh.display">progressbar</option>
<option name="status_indicator_app.status_indicator.colorBy">static_color</option>
<option name="status_indicator_app.status_indicator.fillTarget">text</option>
<option name="status_indicator_app.status_indicator.fixIcon">warning</option>
<option name="status_indicator_app.status_indicator.icon">fix_icon</option>
<option name="status_indicator_app.status_indicator.precision">0</option>
<option name="status_indicator_app.status_indicator.showOption">3</option>
<option name="status_indicator_app.status_indicator.staticColor">#FFFFFF</option>
<option name="status_indicator_app.status_indicator.useColors">true</option>
<option name="status_indicator_app.status_indicator.useThousandSeparator">true</option>
<option name="trellis.enabled">0</option>
<option name="trellis.scales.shared">1</option>
<option name="trellis.size">medium</option>
</viz>
</panel>
</row>
 <row>
<panel>
<title>All</title>
<single>
<search base="index_summary_last">
<query>| stats dc(Name) as Total</query>
</search>
<option name="colorMode">block</option>
<option name="drilldown">none</option>
<option name="height">109</option>
<option name="rangeColors">["0x138D75","0x138D75"]</option>
<option name="rangeValues">[0]</option>
<option name="refresh.display">progressbar</option>
<option name="useColors">1</option>
</single>
</panel>
</row>

thanks in advanced 🙂

[ITWhisperer · Accepted solution]

To be honest, I don't know if this will help you, but you could try saving the job id from the index_summary_last_base base search

    <done>
      <condition>
        <set token="base_sid">$job.sid$</set>
      </condition>
    </done>

Then load the results in subsequent searches

<query>
| loadjob $base_sid$
|search BU=$bu$ AND A $a$ AND B $b$ ....(+ another 14 fields/tokens to search)
</query>

[Od11]

Your solution really helped and made the dashboard loading faster, thank you!

it just caused one little issue that until the base search is loaded - the panels has the message of "Search is waiting for input..." instead of "Waiting for data..." , that can be a little confusing, but I prefer to live with that instead of the slowness.

Thank you very much.