extremely slow dashboard

[mvannini]

hi there,

we developed a custom app to manage CABs (change advisory board).

It uses a couple of collections. one for the configuration (the email address that has to be notifyed) and the CABs itself.

so, the cab_manager lookup, now, is with 25 already processed requests but, since some graphical upgrade, now it has become quite unusable. 

I would even paste or attach the xml if needed (>500 lines), but do you have any clue on what I can try to check as initial ? 

I tryed with claude and gemini that suggested something but didn't helped/solved.

additionally:

the application is deployed on a 3 sh cluster if this can be an issue.

the cab_manager lookup is of 41 fields, tyep string

2 accelleration: cab_id and status. enforced 

there ay be too many token that are to be managed ? 

[bowesmana]

@mvannini Can you paste the XML? There are so many reasons why a dashboard can be slow, so without knowing where to start, it's hard to diagnose.

• Are you using base searches.
• What is the data size you are searching.
• If you look at the job inspector for a slow search what does it tell you

The volume of tokens is not necessarily a problem, I have some dashboards that use a large number and they perform perfectly ok (>300).

Paste the XML using the code block so we can try to help.

[mvannini]

sure, here it is.

Really appreciated. The tracing on the query like suggested didn't report anything strange, there are not so much interactions.

Seems really like  a client problem.

<form version="1.1" theme="dark" hideEdit="false">
  <label>CABs Management (Change Advisory Board) - Enterprise</label>
  <description>Portal for the submission and approval of Splunk Infra changes</description>
  <search id="base_cab_data">
    <query>
      | inputlookup cab_requests_lookup
      | eval status = case(
          like(status, "%CANCELLED%"), "🚫 CANCELLED",
          like(status, "%COMPLETED%") OR status="COMPLETED", "🏁 COMPLETED",
          appr_tech="REJECTED" OR appr_sec="REJECTED" OR appr_soc="REJECTED" OR appr_ecc="REJECTED" OR appr_client="REJECTED", "❌ REJECTED",
          appr_tech="APPROVED" AND appr_sec="APPROVED" AND appr_soc="APPROVED" AND appr_ecc="APPROVED" AND appr_client="APPROVED", "✅ APPROVED",
          1=1, "⏳ PENDING"
        )
    </query>
  </search>
  <fieldset submitButton="false"></fieldset>
  <row>
    <panel>
      <title>🗂️ List of Requested CABs</title>
      <table>
        <search base="base_cab_data">
          <query>| search status IN ("*PENDING", "*APPROVED")
            | eval Tech=if(appr_tech="APPROVED", "✅", if(appr_tech="REJECTED", "❌", "🔲"))
            | eval Sec=if(appr_sec="APPROVED", "✅", if(appr_sec="REJECTED", "❌", "🔲"))
            | eval SOC=if(appr_soc="APPROVED", "✅", if(appr_soc="REJECTED", "❌", "🔲"))
            | eval ECC=if(appr_ecc="APPROVED", "✅", if(appr_ecc="REJECTED", "❌", "🔲"))
            | eval Client=if(appr_client="APPROVED", "✅", if(appr_client="REJECTED", "❌", "🔲"))
            | table cab_id, request_date, objective, requester, status, Tech, Sec, SOC, ECC, Client, implementation_window, estimated_duration
            | sort - request_date</query>
        </search>
        <option name="drilldown">row</option>
        <option name="refresh.display">progressbar</option>
        <drilldown>
          <set token="show_detail">true</set>
          <set token="click_cab_id">$row.cab_id$</set>
        </drilldown>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <title>🗄️ CABs Archive (for consulting purposes)</title>
      <input type="text" token="search_archive" searchWhenChanged="false">
        <label>Search for ID, Requestor o Result:</label>
        <default>*</default>
      </input>
      <table>
        <search base="base_cab_data">
          <query>
            | search cab_id="*$search_archive$*" OR requester="*$search_archive$*" OR outcome="*$search_archive$*"
            | search status IN ("*COMPLETED", "*REJECTED")
            | table cab_id, request_date, objective, status, outcome, issues_encountered, lessons_learned
            | rename outcome as "Final Outcome", issues_encountered as "Issues Encountered", lessons_learned as "Lessons Learned"
            | sort - request_date
          </query>
        </search>
        <option name="count">5</option>
        <option name="drilldown">row</option>
        <drilldown>
          <set token="show_detail">true</set>
          <set token="click_cab_id">$row.cab_id$</set>
        </drilldown>
      </table>
    </panel>
  </row>
  <row depends="$show_detail$">
    <panel>
      <title>🔍 CAB Detail: $click_cab_id$</title>
      <input type="link" token="close_detail_btn" searchWhenChanged="true">
        <label></label>
        <choice value="chiudi">❌ Chiudi Dettaglio</choice>
        <change>
          <condition value="chiudi">
            <unset token="show_detail"></unset>
            <unset token="form.close_detail_btn"></unset>
          </condition>
        </change>
        <default>chiudi</default>
      </input>
      <table>
        <search>
          <query>| inputlookup cab_requests_lookup where cab_id="$click_cab_id$"
            | transpose 0 column_name="Attribute"
            | rename "row 1" as "Value"</query>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <title>👤 1. Basic Information</title>
      <input type="text" token="new_requester">
        <label>Requester (Name/Team)</label>
      </input>
      <input type="dropdown" token="new_severity">
        <label>Severity</label>
        <choice value="LOW">LOW</choice>
        <choice value="MEDIUM">MEDIUM</choice>
        <choice value="HIGH">HIGH</choice>
        <choice value="CRITICAL">CRITICAL (Emergency)</choice>
      </input>
      <input type="multiselect" token="new_env">
        <label>Spoke</label>
        <choice value="All">All Spoke</choice>
        <choice value="EMEA">EMEA</choice>
        <choice value="US">US</choice>
        <choice value="CANADA">CANADA</choice>
        <choice value="CHINA">CHINA</choice>
        <choice value="APAC">APAC</choice>
        <default>All</default>
        <initialValue>All</initialValue>
        <delimiter> OR </delimiter>
      </input>
      <input type="multiselect" token="new_affected_comp">
        <label>Affected Splunk Components</label>
        <choice value="ALL">ALL</choice>
        <choice value="SH">Seach Head</choice>
        <choice value="DP">Deployer</choice>
        <choice value="DS">Deploy Server</choice>
        <choice value="IDX">Indexer</choice>
        <choice value="CM">Cluster Manager</choice>
        <choice value="LM">License Master</choice>
        <choice value="HF">Heavy Forwarder/Intermediate</choice>
        <default>ALL</default>
        <initialValue>ALL</initialValue>
        <delimiter> OR </delimiter>
      </input>
      <input type="text" token="new_emerg_just">
        <label>Emergency Justification (if applicable)</label>
        <default>N/A</default>
      </input>
    </panel>
    <panel>
      <title>🎯 2. Change Description</title>
      <input type="text" token="new_objective">
        <label>Objective</label>
      </input>
      <input type="text" token="new_scope">
        <label>Scope</label>
      </input>
      <input type="text" token="new_impacted_comp">
        <label>Impacted Components</label>
      </input>
      <input type="text" token="new_dependencies">
        <label>Dependencies</label>
        <default>N/A</default>
      </input>
      <input type="text" token="new_stakeholders">
        <label>Impacted Stakeholders</label>
      </input>
    </panel>
  </row>
  <row>
    <panel>
      <title>🛠️ 3. Technical Analysis</title>
      <input type="text" token="new_change_det">
        <label>Change Details</label>
      </input>
      <input type="text" token="new_test_perf">
        <label>Testing Performed</label>
      </input>
      <input type="text" token="new_backup">
        <label>Backup Strategy</label>
      </input>
      <input type="text" token="new_rollback">
        <label>Rollback Plan</label>
      </input>
      <input type="radio" token="new_imp_avail">
        <label>Impact on Service Availability</label>
        <choice value="YES">YES</choice>
        <choice value="NO">NO</choice>
        <default>NO</default>
        <initialValue>NO</initialValue>
      </input>
      <input type="radio" token="new_imp_perf">
        <label>Impact on Performance</label>
        <choice value="YES">YES</choice>
        <choice value="NO">NO</choice>
        <default>NO</default>
        <initialValue>NO</initialValue>
      </input>
    </panel>
    <panel>
      <title>📅 4. Planning &amp; Submit</title>
      <input type="text" token="new_impl_win">
        <label>Proposed Implementation Window (YYYY-MM-DD HH:mm)</label>
      </input>
      <input type="text" token="new_est_dur">
        <label>Estimated Duration (30', 4h, ...)</label>
      </input>
      <input type="text" token="new_exp_down">
        <label>Expected Downtime</label>
      </input>
      <input type="text" token="new_staff">
        <label>Staff Involved</label>
      </input>
      <input type="text" token="new_req_res">
        <label>Required Resources</label>
        <default>N/A</default>
      </input>
      <input type="link" token="submit_cab">
        <label>Azione:</label>
        <choice value="salva">💾 SALVA E INVIA CAB</choice>
        <change>
          <condition value="salva">
            <set token="trigger_insert">true</set>
            <unset token="submit_cab"></unset>
          </condition>
        </change>
      </input>
      <html>
        <br/>
        <hr/>
        <br/>
      </html>
      <table depends="$trigger_insert$">
        <search>
          <done>
            <unset token="trigger_insert"></unset>
          </done>
          <query>| inputlookup cab_requests_lookup
| stats count as total_cabs
| eval next_num = total_cabs + 1

| eval current_year = strftime(now(), "%Y")
| eval padded_num = printf("%03d", next_num)
| eval generated_id = "CAB-" . padded_num . "/" . current_year

| eval request_date = strftime(now(), "%d/%m/%Y %H:%M")
| eval requester="$new_requester$", severity="$new_severity$", environment="$new_env$", emergency_just="$new_emerg_just$", affected_components="$new_affected_comp$"
| eval objective="$new_objective$", scope="$new_scope$", impacted_components="$new_impacted_comp$", dependencies="$new_dependencies$", impacted_stakeholders="$new_stakeholders$"

| eval change_details="$new_change_det$", testing_performed="$new_test_perf$", backup_strategy="$new_backup$", rollback_plan="$new_rollback$", impact_availability="$new_imp_avail$", impact_performance="$new_imp_perf$"
| eval implementation_window="$new_impl_win$", estimated_duration="$new_est_dur$", expected_downtime="$new_exp_down$", staff_involved="$new_staff$", required_resources="$new_req_res$"
| eval outcome="", issues_encountered="", lessons_learned=""

| eval status="PENDING", appr_tech="PENDING", appr_sec="PENDING", appr_soc="PENDING", appr_ecc="PENDING", appr_client="PENDING"
| table generated_id, request_date, requester, severity, environment, emergency_just, affected_components, objective, scope, impacted_components, dependencies, impacted_stakeholders, change_details, testing_performed, backup_strategy, rollback_plan, impact_availability, impact_performance, implementation_window, estimated_duration, expected_downtime, staff_involved, required_resources, outcome, issues_encountered, lessons_learned, status, appr_tech, appr_sec, appr_soc, appr_ecc, appr_client
| rename generated_id as cab_id
| outputlookup append=true cab_requests_lookup</query>
          <earliest>$earliest$</earliest>
          <latest>$latest$</latest>
        </search>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <title>✅ 5A. Approvals (CAB Team Only)</title>
      <input type="dropdown" token="sel_cab_id">
        <label>1. Select CAB ID</label>
        <fieldForLabel>cab_id</fieldForLabel>
        <fieldForValue>cab_id</fieldForValue>
        <search base="base_cab_data">
          <query>| search status="⏳ PENDING" | table cab_id</query>
        </search>
      </input>
      <input type="dropdown" token="sel_role">
        <label>2. Select your Role</label>
        <choice value="appr_tech">Technical Review</choice>
        <choice value="appr_sec">Security Review</choice>
        <choice value="appr_soc">SOC</choice>
        <choice value="appr_ecc">ECC</choice>
        <choice value="appr_client">End Client Review</choice>
      </input>
      <input type="dropdown" token="sel_action">
        <label>3. Decision</label>
        <choice value="APPROVED">🟢 APPROVE</choice>
        <choice value="REJECTED">🔴 REJECT</choice>
      </input>
      <input type="link" token="save_trigger_btn">
        <label>Azione:</label>
        <choice value="execute">✍️ Save Decision</choice>
        <change>
          <condition value="execute">
            <set token="trigger_approval">true</set>
            <unset token="form.save_trigger_btn"></unset>
          </condition>
        </change>
      </input>
      <table depends="$trigger_approval$">
        <search>
          <query>
            | inputlookup cab_requests_lookup where cab_id="$sel_cab_id$"
            | eval _key = _key

            | eval current_user = "$env:user$", current_time = strftime(now(), "%d/%m/%Y %H:%M:%S")
            | eval $sel_role$ = "$sel_action$"
            | eval $sel_role$_user = current_user
            | eval $sel_role$_time = current_time
            | eval status = if(appr_tech="APPROVED" AND appr_sec="APPROVED" AND appr_soc="APPROVED" AND appr_ecc="APPROVED" AND appr_client="APPROVED", "APPROVED", if(appr_tech="REJECTED" OR appr_sec="REJECTED" OR appr_soc="REJECTED" OR appr_ecc="REJECTED" OR appr_client="REJECTED", "REJECTED", "PENDING"))
            | outputlookup cab_requests_lookup append=true key_field=_key
          </query>
          <done>
            <unset token="trigger_approval"></unset>
            <unset token="form.sel_role"></unset>
            <unset token="form.sel_action"></unset>
          </done>
        </search>
        <option name="refresh.display">progressbar</option>
      </table>
      <html depends="$trigger_approval$">
         <div style="color: #55ff55; font-weight: bold; margin-top: 10px;">
           ⏳ Elaborazione salvataggio in corso...
         </div>
      </html>
    </panel>
    <panel>
      <title>📧 5B. Manage Email Address</title>
      <input type="text" token="new_email_app" searchWhenChanged="false">
        <label>New list of Approvers (comma separated)</label>
      </input>
      <input type="text" token="new_email_adm" searchWhenChanged="false">
        <label>New list of Admins (comma separated)</label>
      </input>
      <input type="link" token="save_emails" searchWhenChanged="false">
        <label></label>
        <choice value="save">💾 Sovrascrivi Email</choice>
        <change>
          <condition value="save">
            <set token="trigger_save_emails">true</set>
            <unset token="save_emails"></unset>
          </condition>
        </change>
      </input>
      <html>
        <br/>
        <hr/>
        <br/>
      </html>
      <html>
        <div style="padding: 10px; border-left: 5px solid #4da6ff; background: #333; margin-bottom: 15px;">
          <h4 style="color: #4da6ff;">📧 Configurazione Destinatari Email</h4>
          <p>Visualizza e aggiorna le liste di distribuzione. Questi indirizzi verranno usati in automatico dagli alert di sistema.</p>
        </div>
      </html>
      <table>
        <search>
          <query>| inputlookup cab_config_lookup | table email_approvers, email_admins | rename email_approvers as "Approvers Emails (Queued CABs)", email_admins as "Admin Email (Completed CABs)"</query>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
      <table depends="$trigger_save_emails$">
        <search>
          <query>
            | makeresults
            | eval email_approvers="$new_email_app$", email_admins="$new_email_adm$"
            | table email_approvers, email_admins
            | outputlookup cab_config_lookup
          </query>
          <done>
            <unset token="trigger_save_emails"></unset>
          </done>
        </search>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <title>📝 6. Post Implementation Analysis (Chiusura CAB)</title>
      <input type="dropdown" token="post_cab_id" searchWhenChanged="false">
        <label>1. Select Approved CAB</label>
        <fieldForLabel>cab_id</fieldForLabel>
        <fieldForValue>cab_id</fieldForValue>
        <search base="base_cab_data">
          <query>| search status="*APPROVED" | table cab_id</query>
        </search>
      </input>
      <input type="text" token="post_outcome" searchWhenChanged="false">
        <label>Outcome (Result)</label>
      </input>
      <input type="text" token="post_issues" searchWhenChanged="false">
        <label>Issues Encountered</label>
        <default>Nessuno</default>
      </input>
      <input type="text" token="post_lessons" searchWhenChanged="false">
        <label>Lessons Learned</label>
        <default>N/A</default>
      </input>
      <input type="link" token="submit_post">
        <label>Final Action:</label>
        <choice value="salva">🏁 Registra Esito e Chiudi CAB</choice>
        <change>
          <condition value="salva">
            <set token="trigger_post">true</set>
            <unset token="form.submit_post"></unset>
          </condition>
        </change>
      </input>
      <table depends="$trigger_post$">
        <search>
          <query>
            | inputlookup cab_requests_lookup where cab_id="$post_cab_id$"
            | eval outcome="$post_outcome$"
            | eval issues_encountered="$post_issues$"
            | eval lessons_learned="$post_lessons$"
            | eval status="COMPLETED"
            | outputlookup cab_requests_lookup append=true key_field=_key
          </query>
          <done>
            <unset token="trigger_post"></unset>
            <unset token="form.post_outcome"></unset>
            <unset token="form.post_issues"></unset>
            <unset token="form.post_lessons"></unset>
          </done>
        </search>
        <option name="refresh.display">progressbar</option>
      </table>
      <html>
         <div style="background-color: #1e3f20; padding: 10px; border-radius: 5px; margin-top: 15px;">
           <p style="color: #a8dba8;">
            <i>
              <b>Nota:</b> Salvando l'analisi post-implementazione, lo stato della CAB passerà ad <b>Archiviata (COMPLETED)</b>. L'intervento è ufficialmente concluso.</i>
          </p>
         </div>
      </html>
    </panel>
  </row>
  <row>
    <panel>
      <title>⚙️ 7A. Data cleaning and Removals</title>
      <input type="link" token="trigger_dedup" searchWhenChanged="false">
        <label></label>
        <choice value="clean">🧹 Rimuovi tutti i Duplicati</choice>
        <change>
          <condition value="clean">
            <set token="run_dedup">true</set>
            <unset token="trigger_dedup"></unset>
          </condition>
        </change>
      </input>
      <input type="dropdown" token="del_cab_id" searchWhenChanged="false">
        <label>Select CAB to delete</label>
        <fieldForLabel>cab_id</fieldForLabel>
        <fieldForValue>cab_id</fieldForValue>
        <search base="base_cab_data">
          <query>| table cab_id | dedup cab_id | sort - cab_id</query>
        </search>
      </input>
      <input type="link" token="trigger_delete" searchWhenChanged="false">
        <label></label>
        <choice value="delete">🗑️ Elimina Definitivamente</choice>
        <change>
          <condition value="delete">
            <set token="run_delete">true</set>
            <unset token="trigger_delete"></unset>
          </condition>
        </change>
      </input>
      <html>
        <div style="padding: 10px; border-left: 5px solid #ffcc00; background: #333; margin-bottom: 10px;">
          <h4 style="color: #ffcc00; margin-top: 0;">⚠️ Pulizia Duplicati &amp; Rimozione</h4>
          <p>Usa il tasto Dedup per pulire click multipli. Seleziona un ID per eliminarlo dal database.</p>
        </div>
      </html>
      <table depends="$run_dedup$">
        <search>
          <query>| inputlookup cab_requests_lookup | dedup cab_id | outputlookup cab_requests_lookup</query>
          <done>
            <unset token="run_dedup"></unset>
          </done>
        </search>
      </table>
      <table depends="$run_delete$">
        <search>
          <query>
            | inputlookup cab_requests_lookup where cab_id="$del_cab_id$"
            | eval status="CANCELLED"
            | outputlookup cab_requests_lookup append=true key_field=_key
          </query>
          <done>
            <unset token="run_delete"></unset>
          </done>
        </search>
      </table>
    </panel>
    <panel>
      <title>⚙️ 7B. Forza Cambio Stato</title>
      <input type="dropdown" token="admin_cab_id">
        <label>1. Select CAB ID</label>
        <fieldForLabel>cab_id</fieldForLabel>
        <fieldForValue>cab_id</fieldForValue>
        <search base="base_cab_data">
          <query>| table cab_id | sort - cab_id</query>
        </search>
      </input>
      <input type="dropdown" token="admin_new_status">
        <label>2. Select New Status</label>
        <choice value="PENDING">⚪ PENDING</choice>
        <choice value="APPROVED">🟢 APPROVED</choice>
        <choice value="REJECTED">🔴 REJECTED</choice>
        <choice value="COMPLETED">🏁 COMPLETED</choice>
      </input>
      <input type="link" token="btn_force_status">
        <label>Action:</label>
        <choice value="execute">⚡ Forza Cambio</choice>
        <change>
          <condition value="execute">
            <set token="trigger_admin_status">true</set>
            <unset token="form.btn_force_status"></unset>
          </condition>
        </change>
      </input>
      <html>
        <div style="padding: 10px; border-left: 5px solid #ff9900; background: #333; margin-bottom: 10px;">
          <h4 style="color: #ff9900; margin-top: 0;">⚠️ Override Manuale</h4>
          <p>Bypassa l'iter di approvazione per correggere errori di inserimento.</p>
        </div>
      </html>
      <table depends="$trigger_admin_status$">
        <search>
          <query>
            | inputlookup cab_requests_lookup where cab_id="$admin_cab_id$"
            | eval status="$admin_new_status$"
            | outputlookup cab_requests_lookup append=true key_field=_key
          </query>
          <done>
            <unset token="trigger_admin_status"></unset>
            <unset token="form.admin_cab_id"></unset>
            <unset token="form.admin_new_status"></unset>
          </done>
        </search>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
</form>

[bowesmana]

@mvannini Following on from my previous comment about misplacement of your depends="$token$" pattern.

Your entire dashboard is broken from this point of view. If you create fields in section 1-4 after you fill the last field the new entry is written to the lookup, because the button SALVA E INVIA CAB is ONLY affecting the display of the table.

You will see if you run | inputlookup cab_requests_lookup on its own the new row is written before you click the button.

If you then change a single field, a new CAB is written.

Every time you want to control changes to the lookup, you have to make the <search> dependent on the token and NOT the table.

[bowesmana]

One thing that stands out is your deduplication of the CAB table.  In this search (line 462)

      <table depends="$run_dedup$">
        <search>
          <query>| inputlookup cab_requests_lookup | dedup cab_id | outputlookup cab_requests_lookup</query>
          <done>
            <unset token="run_dedup"></unset>
          </done>
        </search>
      </table>

your <table> has a depends statement, but that will NOT stop the search from running, so the first thing that happens when you load the dashboard, is that you totally rewrite the KV store and this happens while the other searches are running, so there is a concurrency issue going on in the KV store reading the data in your base search as well as re-writing it in the above.

You should make the <search> dependent on the token, i.e.

<search depends="$run_dedup$">

or you could do it this way

      <table depends="$never_show$">
        <search>
          <query>| inputlookup cab_requests_lookup | ``` $run_dedup$ ``` dedup cab_id | outputlookup cab_requests_lookup</query>
          <done>
            <unset token="run_dedup"></unset>
          </done>
        </search>
      </table>

which embeds the token in a comment in the SPL itself, which will then cause the search to run each time the token is set and it never shows the table.

I believe that is what you want given the input at 7A. At the moment, you see nothing when the table loads, but the kv is rewritten, then if you select the clean option in 7A, you will see the table appear then disappear when the search runs again.

I only tested this with a CSV, not KV store, but can't reproduce the duplicate searches seen by @ITWhisperer 

That may play into why the searches are running more than once, I am not sure how the KV store handles concurrent read/write on the lookup, but change that and see how that affects things.

[mvannini]

the duplication removal action was needed since some time we discovered that records were duplicated due to the missed refresh of the entire view expecially just after an insert, but in effect inow it can even be removed having learned how to handle it.

Anywhay the solutions has been acquired. We will review with those suggestions.

Thank you very much.

[ITWhisperer]

Make the updating search "dependent" on a token which is then unset in the done handler of the search.

To make the execution of the search "dependent" rather than just the display of the table, either include an assignment (eval) to a dummy field which you can remove with the fields command, or include the token in a comment (Thanks to @bowesmana for the suggestion here).

[ITWhisperer]

It may depend on which version of Splunk you are running.

I recreated your dashboard and added some trace functionality and it appears that the searches are being executed multiple times, which will make it appear slow. However, looking at the source, it is not immediately obvious to me why the searches are running so many times. It may be due to so many references to the base search, but I could be wrong.

Rather than using a base search, you could try saving the $job.sid$ in a token, and then use loadjob to retrieve the results. This does have the possibility that the loadjob fails if the results have expired, or are on a different search head in a cluster. This would just mean that the dashboard needs to be reloaded, or you could add some sort of refresh button.

[mvannini]

Even this change is interesting. 

will try.

Thank you.

[mvannini]

you are right, a bit of mess 🙂

the collections, btw, aren't kv  😉 ? mine are kv. 

it's all with SPL without any custom commands. 

the graphical upgrade refers to panels that are shown/hidden when a line is selected in order to see the details but the strange is that is slowered not only in that part but everywhere, even when selecting item from a multi dropdown (static values) and you decide to remove the ALL (the default), for example, it takes at least 1'.
well, downgrading it's an option but it possibile that we cannot build something nice and without so much customization/personalization/js/other ... ? 

[ITWhisperer]

(By the way, I have renamed your post as it looks like there was a typo and you meant "slow" not "low".)

I am assuming (since you mentioned XML), that you are using Classic dashboards, not Dashboard Studio?

Given this assumption, it is likely that you have a lot of tokens being used in various searches. These searches will execute whenever a dependent token is changed. Without careful management of these token updates, you may find (and are probably experiencing) that searches run multiple times even when you thought they would only execute once.

It is very difficult to advise for your specific usecase without sight of the source, but you may be able to control when searches execute by using additional tokens (I know this sounds counter-intuitive) which are only set when a search completes, using a done handler. In this way, you can enforce the order in which searches are executed.

Another check that you may be able to do with your dashboard, is to temporarily introduce a trace function to see when searches are executed so you can figure out which ones might need changing. I have done this sort of thing before by using appendpipe and outputlookup to store the trace in a csv file for later analysis. There may be other ways to achieve this with other browser/js tools.

[mvannini]

yes, Classic Dashboard.

This sound interesting. Going to study on how to implement it

Thank you

[ITWhisperer]

It is not clear what is going on here:

Is your "custom app" just a place holder for the dashboard and lookup files?

Are the lookups csv files or kv stores?

Is your processing done entirely in SPL or do you have custom commands (in your app)?

What was the nature of the "graphical upgrade", and in what way has it affected your app/dashboard?

Can you downgrade/rollback the upgrade to fix the issue?