topic How to remove a label that's not being searched using token? in Dashboards & Visualizations

How to remove a label that's not being searched using token?

vino06 — Wed, 16 Aug 2017 10:47:19 GMT

Hi,

I'm having difficulties removing the "CPU Used" label on my dashboard. When I search for "Memory Used" or "Free Space," the "CPU Used" label shows on the graph. I have 2 appendcols for each search; you may see my search below and also the attached screenshot.

index=perfmon eventtype="perfmon_windows" * Host="$hostname$" object=Processor counter="$kpi$" 
 | multikv 
 | eval cpu=100-Value
 | timechart span=5m latest(cpu)
 | rename latest(cpu) as "CPU Used"

 | append [search index=perfmon eventtype="perfmon_windows" * Host="$hostname$" object=Memory counter="$kpi$" 
 | multikv 
 | timechart span=5m latest(Value)
 | rename latest(Value) as "Memory Used"]

| append [search index=perfmon Host="$hostname$"  sourcetype="Perfmon:LogicalDisk" object=LogicalDisk counter="$kpi$" mount!=HarddiskVolume*
| multikv
| timechart span=5m latest(storage_free_percent) as "Free Space" by mount]

Thank you!
Kevin

Re: How to remove a label that's not being searched using token?

niketn — Wed, 16 Aug 2017 11:41:00 GMT

@vino06, Based on your screenshots I dont think you need to run append or appendcols since you are displaying only one KPI at a time.

Code the KPI dropdown change event to create the query specific to a KPI.

    <input type="dropdown" token="tokKPI">
      <label>KPI</label>
      <choice value="CPU">CPU</choice>
      <choice value="Memory">Memory</choice>
      <choice value="% Free Space">Free Space</choice>
      <change>
        <condition label="CPU">
          <set token="tokQuery">index="perfmon" eventtype="perfmon_windows" Host="$hostname$" object="Processor" counter="$value$" 
  | multikv 
  | eval cpu=100-Value
  | timechart span=5m latest(cpu)
  | rename latest(cpu) as "CPU Used"</set>
        </condition>
        <condition label="Memory">
          <set token="tokQuery">index="perfmon" eventtype="perfmon_windows" Host="$hostname$" object="Memory" counter="$value$" 
| multikv 
| timechart span=5m latest(Value)
| rename latest(Value) as "Memory Used"</set>
        </condition>
        <condition label="Free Space">
          <set token="tokQuery">index="perfmon" Host="$hostname$" sourcetype="Perfmon:LogicalDisk" object=LogicalDisk counter="$value$" mount!="HarddiskVolume*"
| multikv
| timechart span=5m latest(storage_free_percent) as "Free Space" by mount</set>
        </condition>
      </change>
      <default>CPU</default>
    </input>

Then use the KPI token $tokKPI$ as your search query

  <row>
    <panel>
      <chart>
        <search>
          <query>$tokQuery$</query>
        </search>
        <option name="charting.chart">line</option>
      </chart>
    </panel>
  </row>

Re: How to remove a label that's not being searched using token?

vino06 — Wed, 16 Aug 2017 12:18:18 GMT

@niketnilay How about the query for my Hostname? do I need to change it also? my KPI are connected to my Hostname dropdown.

Re: How to remove a label that's not being searched using token?

niketn — Wed, 16 Aug 2017 18:53:50 GMT

On change of hostname you can unset the token $tokKPI$ and $form.tokKPI$, so that KPI would be selected after changing the host name

<change>
    <unset token="tokKPI"></unset>
    <unset token="form.tokKPI"></unset>
</change>

Re: How to remove a label that's not being searched using token?

vino06 — Fri, 18 Aug 2017 08:02:55 GMT

@niketnilay I tried to use the search you gave me but its not working and I find it difficult. Is there a simple way? I think my search is just wrong and need to be fix.

Re: How to remove a label that's not being searched using token?

niketn — Fri, 18 Aug 2017 09:41:16 GMT

Based on your initial question and screenshot, I had not changed anything in your query since I assumed your queries were working fine as per the screenshot.

Since you had not provided Performance Counters (KPIs) in your question, I had just cooked up the KPI Values. Please double check to add specific counter values in the dropdown. I expect following to be the counters you are using. However, unfortunately I can not be sure.

   <choice value="% Processor Time">CPU</choice>
   <choice value="% Committed Bytes In Use">Memory</choice>
   <choice value="% Free Space">Free Space</choice>

Run the queries in separate searches for specific KPI and test. Also you can rename field name in timechart command itself. Further for CPU KPI you should ideally perform 100-CPU% after the timechart command.

 index=perfmon eventtype="perfmon_windows" * Host="$hostname$" object=Processor counter="$kpi$" 
  | timechart span=5m latest(Value) as "CPU Used"
  | eval "CPU Used"=100-'CPU Used'

Please let me know if you are facing any issue other than this or if the dropdown token is not working as expected.

Re: How to remove a label that's not being searched using token?

vino06 — Fri, 18 Aug 2017 10:03:37 GMT

@niketnilay, Here is my xml for the drop down.

      <input type="dropdown" token="hostname">
        <label>Hostname</label>
        <search>
          <query>index=perfmon Host=* | dedup Host | sort 0 Host</query>
          <earliest>-15m</earliest>
          <latest>now</latest>
        </search>
        <fieldForLabel>Host</fieldForLabel>
        <fieldForValue>Host</fieldForValue>
      </input>
      <input type="dropdown" token="kpi" searchWhenChanged="false">
        <label>KPI</label>
        <choice value="% Idle Time">CPU</choice>
        <choice value="% Committed Bytes In Use">Memory</choice>
        <choice value="% Free Space">Free Space</choice>
        <search>
          <query></query>
          <earliest>-15m</earliest>
          <latest>now</latest>
        </search>
        <fieldForLabel>counter</fieldForLabel>
        <fieldForValue>counter</fieldForValue>
      </input>
      <chart>
        <search>
          <query>index=perfmon Host="$hostname$" object=Processor counter="$kpi$" 
 | eval cpu=100-Value
 | timechart span=5m latest(cpu) as "CPU Used"

 | append [search index=perfmon Host="$hostname$" object=Memory counter="$kpi$" 
 | timechart span=5m latest(Value) as "Memory Used"]

 | append [search index=perfmon Host="$hostname$"  sourcetype="Perfmon:LogicalDisk" object=LogicalDisk counter="$kpi$" mount!=HarddiskVolume*
| timechart span=5m latest(storage_free_percent) as "Free Space" by mount]
          <earliest>$timepckr.earliest$</earliest>
          <latest>$timepckr.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
        <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
        <option name="charting.axisTitleX.visibility">visible</option>
        <option name="charting.axisTitleY.visibility">visible</option>
        <option name="charting.axisTitleY2.visibility">visible</option>
        <option name="charting.axisX.scale">linear</option>
        <option name="charting.axisY.scale">linear</option>
        <option name="charting.axisY2.enabled">0</option>
        <option name="charting.axisY2.scale">log</option>
        <option name="charting.chart">line</option>
        <option name="charting.chart.bubbleMaximumSize">50</option>
        <option name="charting.chart.bubbleMinimumSize">10</option>
        <option name="charting.chart.bubbleSizeBy">area</option>
        <option name="charting.chart.nullValueMode">connect</option>
        <option name="charting.chart.showDataLabels">none</option>
        <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
        <option name="charting.chart.stackMode">default</option>
        <option name="charting.chart.style">shiny</option>
        <option name="charting.drilldown">none</option>
        <option name="charting.layout.splitSeries">0</option>
        <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
        <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
        <option name="charting.legend.placement">right</option>
      </chart>
    </panel>

Re: How to remove a label that's not being searched using token?

niketn — Tue, 29 Sep 2020 15:27:30 GMT

I can see that you have not made changes to KPI Dropdown as request to pass on Single Search query instead of append.

Please run splunk search on index="perfmon" earliest="-15m" latest="now" to pull sample events to validate all fields like :
1) sourcetypes for different Performance Counters
2) Host field name is Host or host? I expected it to be lower case. Please confirm.
3) Object Collection and Counter field names
4) Other fields like mount and storage_free_percent, whether they actually exist for "Logical Disc" performance counters.

  <row>
    <panel>
      <input type="time" token="timepckr" searchWhenChanged="true">
        <label>Select Time</label>
        <default>
          <earliest>-15m</earliest>
          <latest>now</latest>
        </default>
      </input>
      <input type="dropdown" token="hostname" searchWhenChanged="true">
        <label>Hostname</label>
        <choice value="*">All</choice>
        <default>*</default>
        <search>
          <query>index=performance_counters sourcetype="Perfmon:*" host="*" earliest="$timepckr.earliest$" latest="$timepckr.latest$"
| dedup host
| table host
| sort 0 host</query>
        </search>
        <fieldForLabel>host</fieldForLabel>
        <fieldForValue>host</fieldForValue>
      </input>
      <input type="dropdown" token="kpi" searchWhenChanged="true">
        <label>KPI</label>
        <selectFirstChoice>true</selectFirstChoice>
        <search>
          <query>index=performance_counters sourcetype="Perfmon:*" host="$hostname$" object="*" counter="*" earliest="$timepckr.earliest$" latest="$timepckr.latest$"
| dedup counter
| table counter collection
| sort counter</query>
        </search>
        <fieldForLabel>collection</fieldForLabel>
        <fieldForValue>counter</fieldForValue>
        <change>
          <condition label="CPU">
            <set token="tokQuery">index=performance_counters sourcetype="Perfmon:CPU" host="$hostname$" object="Processor Information" counter="$value$" 
| timechart span=5m latest(Value) as "CPU Used"</set>
          </condition>
          <condition label="Memory">
            <set token="tokQuery">index=performance_counters sourcetype="Perfmon:Memory" host="$hostname$" object="Memory" counter="$value$" 
| timechart span=5m latest(Value) as "Memory Used"</set>
          </condition>
          <condition label="Free Space">
            <set token="tokQuery">index=performance_counters host="$hostname$" sourcetype="Perfmon:LogicalDisk" object="LogicalDisk" mount!="HarddiskVolume*" counter="$value$"
 | timechart span=5m latest(storage_free_percent) as "Free Space" by mount</set>
          </condition>
          <condition>
            <set token="tokQuery">| makeresults| eval ErrorMsg="Performance Counter not configured"</set>
          </condition>
        </change>
      </input>
      <chart>
        <search>
          <query>$tokQuery$</query>
          <earliest>$timepckr.earliest$</earliest>
          <latest>$timepckr.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
        <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
        <option name="charting.axisTitleX.visibility">visible</option>
        <option name="charting.axisTitleY.visibility">visible</option>
        <option name="charting.axisTitleY2.visibility">visible</option>
        <option name="charting.axisX.scale">linear</option>
        <option name="charting.axisY.scale">linear</option>
        <option name="charting.axisY2.enabled">0</option>
        <option name="charting.axisY2.scale">log</option>
        <option name="charting.chart">line</option>
        <option name="charting.chart.bubbleMaximumSize">50</option>
        <option name="charting.chart.bubbleMinimumSize">10</option>
        <option name="charting.chart.bubbleSizeBy">area</option>
        <option name="charting.chart.nullValueMode">connect</option>
        <option name="charting.chart.showDataLabels">none</option>
        <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
        <option name="charting.chart.stackMode">default</option>
        <option name="charting.chart.style">shiny</option>
        <option name="charting.drilldown">none</option>
        <option name="charting.layout.splitSeries">0</option>
        <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option>
        <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
        <option name="charting.legend.placement">right</option>
      </chart>
    </panel>
  </row>

Following are some of the major changes:
1) Process performance counter changed to "% Processor Time". This way extra calculation is not required in query.
2) earliest and latest moved to dropdown query to fetch the records from same timeframe being instead of hard-coded 15 min for Dashboard Search queries including dropdown searches.
3) Sourcetype added to all searches base query.

There is still scope of performance improvement through:
1) If host name is being pulled from host and not Host, tstats can be used to pull host name for dropdown.
2) Running single base search using Post Processing to fetch required details in single query. Then show only fields interested in.