drilldown and panel depend not working when clicki...

dickersons · ‎03-15-2025

Hi,

I am doing an initial search based off of initial field inputs within a dashboard. The issue I am having is after my chart gets populated with standard deviation, i am attempting to do a drilldown click on the chart and once that action happens another panel dynamically appears with the log events from the date/time data point from my chart. Unfortunately this is not working the panel is always displayed and does a search. No matter the data point I click in the chart the search happens but doesn't use the date/time of the click. Even my "labelApp" token is not displaying properly. See below:

<form version="1.1">
  <label>API Gateway Dynamic Application Reporting</label>
  <!--<row>-->
  <!--  <panel>-->
  <!--    <title>THESE ARE MY TOKEN VALUES</title>-->
  <!--    <html>-->
  <!--      <h2>Index = $indexName$</h2>-->
  <!--      <h2>Cluster = $clusterName$</h2>-->
  <!--      <h2>SourceType = mule:app:app</h2>-->
  <!--      <h2>Application = $labelApp$</h2>-->
  <!--      <h2>ErrorSearch = $errorSearch$</h2>-->
  <!--      <h2>Time = $searchTime$</h2>-->
  <!--      <h2>drilldown1 = $earliest$</h2>-->
  <!--      <h2>drilldown2 = $latest$</h2>-->
  <!--    </html>-->
  <!--  </panel>-->
  <!--</row>-->
  <search id="baseSearch">
    <query>index=$indexName$ cluster_name=$clusterName$ sourcetype=mule:app:app label_app=$labelApp$ ("\"statusCode\"") | rex .*\"traceId\"\s:\s\"?(?&lt;traceId&gt;.*?)\".* | rex "(?s)\"statusCode\"\s:\s\"?(?&lt;statusCode&gt;[245]\d{2})\"?" | stats count by statusCode | eventstats sum(count) as totalCount | eval percentage=round(count*100/totalCount,3) </query>
    <earliest>$searchTime.earliest$</earliest>
    <latest>$searchTime.latest$</latest>
  </search>
  <search id="baseSearch2">
    <query>index=$indexName$ cluster_name=$clusterName$ sourcetype=mule:app:app label_app=$labelApp$ ("\"statusCode\"") | rex .*\"traceId\"\s:\s\"?(?&lt;traceId&gt;.*?)\".* | rex "(?s)\"statusCode\"\s:\s\"?(?&lt;statusCode&gt;[245]\d{2})\"?" | timechart span=1$timeSpan$ count(statusCode) as "Number_Of_Requests" | eventstats mean(Number_Of_Requests) as "Average_Requests_Per_Time_Span" stdev(Number_Of_Requests) as "Standard_Deviation" | eval Standard_Deviation=round(Standard_Deviation,2) | eval Average_Requests_Per_Time_Span=round(Average_Requests_Per_Time_Span,2)</query>
    <earliest>$searchTime.earliest$</earliest>
    <latest>$searchTime.latest$</latest>
  </search>
  <fieldset submitButton="false" autoRun="false">
    <input type="radio" token="indexName">
      <label>Index</label>
      <choice value="br_master_application_non-prod">UAT</choice>
      <choice value="br_master_application_prod">Prod</choice>
      <change>
        <condition value="br_master_application_non-prod">
          <set token="clusterName">"broadridge-msapi-gateway-proxy-uatcluster"</set>
        </condition>
        <condition value="br_master_application_prod">
          <set token="clusterName">"broadridge-msapi-gateway-proxy-prdcluster"</set>
        </condition>
      </change>
      <search>
        <query/>
        <earliest>-24h@h</earliest>
        <latest>now</latest>
      </search>
    </input>
    <input type="radio" token="timeSpan">
      <label>Time_Span</label>
      <choice value="s">Second</choice>
      <choice value="m">Minute</choice>
      <choice value="h">Hour</choice>
      <choice value="d">Day</choice>
    </input>
    <input type="dropdown" token="labelApp" depends="$indexName$" searchWhenChanged="true">
      <label>Application</label>
      <choice value="*">All</choice>
      <default>*</default>
      <fieldForLabel>label_app</fieldForLabel>
      <fieldForValue>label_app</fieldForValue>
      <search>
        <query>index=$indexName$ cluster_name=$clusterName$ sourcetype=mule:app:app label_app=* | dedup label_app | table label_app | sort label_app</query>
        <earliest>-24h@h</earliest>
        <latest>now</latest>
      </search>
    </input>
    <input type="time" token="searchTime" searchWhenChanged="true">
      <label>Time</label>
      <default>
        <earliest>-7d@d</earliest>
        <latest>now</latest>
      </default>
    </input>
  </fieldset>
  <row>
    <panel>
      <chart>
        <title>Status Code By Slice ($labelApp$)</title>
        <search base="baseSearch">
          <query>| fields - count totalCount | chart max(percentage) by statusCode</query>
          <!--<earliest>$searchTime.earliest$</earliest>-->
          <!--<latest>$searchTime.latest$</latest>-->
        </search>
        <option name="charting.chart">pie</option>
        <option name="charting.drilldown">none</option>
        <option name="charting.chart.showLabels">true</option>
        <option name="charting.chart.showPercent">true</option>
      </chart>
    </panel>
    <panel>
      <table>
        <title>All Status Code Percentage Table ($labelApp$)</title>
        <search base="baseSearch">
          <query>| table statusCode, count, totalCount, percentage</query>
          <!--<earliest>$searchTime.earliest$</earliest>-->
          <!--<latest>$searchTime.latest$</latest>-->
        </search>
        <option name="drilldown">none</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <chart>
        <title>Total Combined Requests Per Time_Span Graph ($labelApp$)</title>
        <search base="baseSearch2">
          <query/>
          <!--<earliest>$searchTime.earliest$</earliest>-->
          <!--<latest>$searchTime.latest$</latest>-->
        </search>
        <option name="charting.axisLabelsX.majorLabelStyle.rotation">-45</option>
        <option name="charting.axisTitleX.visibility">visible</option>
        <option name="charting.axisTitleY.visibility">visible</option>
        <option name="charting.axisTitleY2.visibility">visible</option>
        <option name="charting.chart">line</option>
        <option name="charting.chart.resultTruncationLimit">500000</option>
        <option name="charting.data.count">500000</option>
        <option name="charting.chart.overlayFields">Deviation,"Average Requests Per Time_Span"</option>
        <option name="charting.drilldown">all</option>
        <option name="charting.legend.placement">bottom</option>
        <option name="refresh.display">preview</option>
        <drilldown>
          <eval token="drilldown1">$earliest$</eval>
          <eval token="drilldown2">$latest$</eval>
        </drilldown>
      </chart>
    </panel>
    <panel>
      <table>
        <title>Total, Average, and Standard Deviation Requests Per Time_Span Table ($labelApp$)</title>
        <search base="baseSearch2">
          <query/>
          <!--<earliest>$searchTime.earliest$</earliest>-->
          <!--<latest>$searchTime.latest$</latest>-->
        </search>
        <option name="drilldown">none</option>
      </table>
    </panel>
  </row>
  <row>
      <panel>
      <title>Latency Metrics for Trade Execution</title>
      <table>
          <search>
              <query>
                  index=$indexName$ sourcetype="mule:app:app" 
                  aws_account_name="CORP-MSAPIGW" label_app=$label_app$
                  | rex "traceId=\"(?&lt;trace_id>[^\"]+)\""
                  | rex "clientId=\"(?&lt;client_id>[^\"]+)\""
                  | rex "message=\"(?&lt;message>[^\"]+)\""
                  | rex "request_method=\"(?&lt;request_method>[^\"]+)\""
                  | rex "request_url=\"(?&lt;request_url>[^\"]+)\""
                  | rex "request_queryParams_account=\"(?&lt;account>[^\"]+)\""
                  | rex "request_headers_x-request-id=\"(?&lt;x_request_id>[^\"]+)\""
                  | rex "statusCode=\"(?&lt;status_code>\d+)\""
                  | rex "latency_backend_latency_in_ms=\"(?&lt;backend_latency>[0-9]+)\""
                  | rex "latency_request_latency_in_ms=\"(?&lt;request_latency>[0-9]+)\""
                  | rex "latency_response_latency_in_ms=\"(?&lt;response_latency>[0-9]+)\""
                  | eval backend_latency_ms=tonumber(backend_latency), 
                         request_latency_ms=tonumber(request_latency), 
                         response_latency_ms=tonumber(response_latency)
                  | eval total_latency_ms = backend_latency_ms + request_latency_ms + response_latency_ms
                  | eventstats perc90(total_latency_ms) as perc90_threshold
                  | where total_latency_ms &lt;= perc90_threshold
                  | eventstats avg(backend_latency_ms) as avg_backend_latency_ms, 
                               avg(request_latency_ms) as avg_request_latency_ms, 
                               avg(response_latency_ms) as avg_response_latency_ms
                  | eval avg_90_percent_latency_ms = avg_backend_latency_ms + avg_request_latency_ms + avg_response_latency_ms
                  | rename backend_latency_ms AS "Backend Latency (ms)", 
                           request_latency_ms AS "Request Latency (ms)", 
                           response_latency_ms AS "Response Latency (ms)", 
                           total_latency_ms AS "Total Latency (ms)", 
                           avg_90_percent_latency_ms AS "90% Avg Total Latency (ms)"
                  | table trace_id, client_id, message, request_method, request_url, account, x_request_id, status_code, 
                          "Backend Latency (ms)", "Request Latency (ms)", "Response Latency (ms)", 
                          "Total Latency (ms)", "90% Avg Total Latency (ms)"
              </query>
          </search>
            <drilldown>
              <eval token="drilldown1">$earliest$</eval>
              <eval token="drilldown2">$latest$</eval>
            </drilldown>
      </table>
  </panel>
  </row>
  <row depends="$drilldown1$">
    <panel>
      <event>
        <title>Drill Down Events</title>
        <search>
          <query>index=$indexName$ cluster_name=$clusterName$ sourcetype=mule:app:app label_app=$labelApp$ ("\"statusCode\"")
          </query>
          <earliest>$drilldown1$</earliest>
          <latest>$drilldown2$</latest>
        </search>
        <option name="list.drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </event>
    </panel>
  </row>
</form>

I commented out the last part as I have not gotten to that piece of the dashboard yet. Any help would be greatly appreciated as I have been banging my head on this for a day or more at this point.

yuanliu · ‎03-17-2025

Unfortunately this is not working the panel is always displayed and does a search. No matter the data point I click in the chart the search happens but doesn't use the date/time of the click. Even my "labelApp" token is not displaying properly. See below:

Three corrections:

The panel you are trying to hide is "Drill Down Events". It does not appear before you select an index. In fact, it does not appear after you select an index to populate the panel titled "Total Combined Requests Per Time_Span Graph ($labelApp$)", either. No matter what you do, this panel will never appear because the token $drilldown1$ can never be populated in this code.
$labelApp$ IS displayed properly in that panel's title. When selection is "All", it displays an asterisk ("*"). This is the value of "All".
No matter what datapoint you click, token $drilldown1$ cannot be populated based on your drilldown logic, <eval token="drilldown1">$earliest$</eval>. The search used in "Total Combined Requests Per Time_Span Graph ($labelApp$)", namely baseSearch2, does not produce a field named earliest. This is why nothing can happen when you click in that panel.

drilldown and panel depend not working when clicking chart

chart

other

.conf25 Global Broadcast: Don’t Miss a Moment

Observe and Secure All Apps with Splunk

What's New in Splunk Observability - August 2025

Are you a member of the Splunk Community?