topic Re: Issue with the search base in Dashboards & Visualizations

Issue with the search base

anjibabu — Thu, 16 Apr 2020 12:13:05 GMT

hi Friends,

below are my queries,

index=perfmon source="Perfmon:LogicalDisk" counter="% Free Space" | search host = DMOPWMD1PDDB0* | eval FreeSpace =100-( Value ) | stats min(FreeSpace) as hostavg by host,instance | table host,instance,hostavg | chart min(hostavg) by host,instance
index=perfmon sourcetype="Perfmon:Memory" counter="% Committed Bytes In Use" | search host = DMOPWMD1PDDB0* | timechart perc90(Value) by host limit=0 span=1m

i created the below search id's for the search and created the panels, these are working fine in search, but not working in dashboard\panels, the panels are showing "No Results", could you please advise.

search id's:

index=perfmon source="Perfmon:LogicalDisk" counter="% Free Space"
$TimeRangePkr.earliest$
$TimeRangePkr.latest$
5m
delay

true

<query>index=perfmon sourcetype="Perfmon:Memory" counter="% Committed Bytes In Use"</query>
<earliest>$TimeRangePkr.earliest$</earliest>
<latest>$TimeRangePkr.latest$</latest>
<refresh>5m</refresh>
<refreshType>delay</refreshType>
<progress>
  <set token="show_html">true</set>
</progress>
<done>
  <unset token="show_html"></unset>
</done>

Panels:

<panel>
  <chart>
    <title>DISK%</title>
    <search base="Disk1">
      <query>| search host = DMOPWMD1PDDB0* | eval FreeSpace =100-( Value )  | stats min(FreeSpace) as hostavg by host,instance | table host,instance,hostavg | chart min(hostavg) by host,instance</query>
      </search>
    <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
    <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
    <option name="charting.axisTitleX.text">TIME</option>
    <option name="charting.axisTitleX.visibility">visible</option>
    <option name="charting.axisTitleY.text">HOST</option>
    <option name="charting.axisTitleY.visibility">visible</option>
    <option name="charting.axisTitleY2.visibility">visible</option>
    <option name="charting.axisX.scale">linear</option>
    <option name="charting.axisY.scale">linear</option>
    <option name="charting.axisY2.enabled">0</option>
    <option name="charting.axisY2.scale">inherit</option>
    <option name="charting.chart">column</option>
    <option name="charting.chart.bubbleMaximumSize">50</option>
    <option name="charting.chart.bubbleMinimumSize">10</option>
    <option name="charting.chart.bubbleSizeBy">area</option>
    <option name="charting.chart.nullValueMode">gaps</option>
    <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
    <option name="charting.chart.stackMode">default</option>
    <option name="charting.chart.style">shiny</option>
    <option name="charting.drilldown">all</option>
    <option name="charting.layout.splitSeries">0</option>
    <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
    <option name="charting.legend.placement">right</option>
    <option name="refresh.display">progressbar</option>
  </chart>
</panel>

<panel>
  <chart>
    <title>MEMORY%</title>
    <search base="Mem">
      <query>| search host = DMOPWMD1PDDB0* | timechart perc90(Value) by host limit=0 span=1m</query>
      </search>
    <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
    <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
    <option name="charting.axisTitleX.text">TIME</option>
    <option name="charting.axisTitleX.visibility">visible</option>
    <option name="charting.axisTitleY.text">HOST</option>
    <option name="charting.axisTitleY.visibility">visible</option>
    <option name="charting.axisTitleY2.visibility">visible</option>
    <option name="charting.axisX.scale">linear</option>
    <option name="charting.axisY.scale">linear</option>
    <option name="charting.axisY2.enabled">0</option>
    <option name="charting.axisY2.scale">inherit</option>
    <option name="charting.chart">line</option>
    <option name="charting.chart.bubbleMaximumSize">50</option>
    <option name="charting.chart.bubbleMinimumSize">10</option>
    <option name="charting.chart.bubbleSizeBy">area</option>
    <option name="charting.chart.nullValueMode">gaps</option>
    <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
    <option name="charting.chart.stackMode">default</option>
    <option name="charting.chart.style">shiny</option>
    <option name="charting.drilldown">all</option>
    <option name="charting.layout.splitSeries">0</option>
    <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
    <option name="charting.legend.placement">right</option>
    <option name="refresh.display">progressbar</option>
  </chart>
</panel>

Re: Issue with the search base

mayurr98 — Thu, 16 Apr 2020 16:32:24 GMT

Can you paste your entire dashboard XML i.e source code?

Re: Issue with the search base

anjibabu — Thu, 16 Apr 2020 23:57:50 GMT

DB Performance Clone_4/14

index=perfmon source="Perfmon:LogicalDisk" counter="% Free Space"
$TimeRangePkr.earliest$
$TimeRangePkr.latest$
5m
delay

<query>index=perfmon sourcetype="Perfmon:Memory" counter="% Committed Bytes In Use"</query>
<earliest>$TimeRangePkr.earliest$</earliest>
<latest>$TimeRangePkr.latest$</latest>
<refresh>5m</refresh>
<refreshType>delay</refreshType>


<input type="time" token="TimeRangePkr" searchWhenChanged="true">
  <label>Time Range</label>
  <default>
    <earliest>-60m@m</earliest>
    <latest>now</latest>
  </default>
</input>
<input type="checkbox" token="Global">
  <label></label>
  <choice value="Global">Global</choice>
</input>
<input type="checkbox" token="Reporting">
  <label></label>
  <choice value="Reporting">Reporting</choice>
</input>
 </fieldset>

<panel>
  <chart>
    <title>DISK%</title>
    <search base="Disk1">
      <query>search host = DMOPWMD1PDDB0* | eval FreeSpace =100-( Value )  | stats min(FreeSpace) as hostavg by host,instance | table host,instance,hostavg | chart min(hostavg) by host,instance</query>
    </search>
    <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
    <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
    <option name="charting.axisTitleX.text">TIME</option>
    <option name="charting.axisTitleX.visibility">visible</option>
    <option name="charting.axisTitleY.text">HOST</option>
    <option name="charting.axisTitleY.visibility">visible</option>
    <option name="charting.axisTitleY2.visibility">visible</option>
    <option name="charting.axisX.scale">linear</option>
    <option name="charting.axisY.scale">linear</option>
    <option name="charting.axisY2.enabled">0</option>
    <option name="charting.axisY2.scale">inherit</option>
    <option name="charting.chart">column</option>
    <option name="charting.chart.bubbleMaximumSize">50</option>
    <option name="charting.chart.bubbleMinimumSize">10</option>
    <option name="charting.chart.bubbleSizeBy">area</option>
    <option name="charting.chart.nullValueMode">gaps</option>
    <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
    <option name="charting.chart.stackMode">default</option>
    <option name="charting.chart.style">shiny</option>
    <option name="charting.drilldown">all</option>
    <option name="charting.layout.splitSeries">0</option>
    <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
    <option name="charting.legend.placement">right</option>
    <option name="refresh.display">progressbar</option>
  </chart>
</panel>
<panel>
  <chart>
    <title>MEMORY%</title>
    <search base="Mem">
      <query>search host=DMOPWMD1PDDB0* | timechart perc95(Value) by host limit=0 span=1m</query>
      </search>
    <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
    <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
    <option name="charting.axisTitleX.text">TIME</option>
    <option name="charting.axisTitleX.visibility">visible</option>
    <option name="charting.axisTitleY.text">HOST</option>
    <option name="charting.axisTitleY.visibility">visible</option>
    <option name="charting.axisTitleY2.visibility">visible</option>
    <option name="charting.axisX.scale">linear</option>
    <option name="charting.axisY.scale">linear</option>
    <option name="charting.axisY2.enabled">0</option>
    <option name="charting.axisY2.scale">inherit</option>
    <option name="charting.chart">line</option>
    <option name="charting.chart.bubbleMaximumSize">50</option>
    <option name="charting.chart.bubbleMinimumSize">10</option>
    <option name="charting.chart.bubbleSizeBy">area</option>
    <option name="charting.chart.nullValueMode">gaps</option>
    <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
    <option name="charting.chart.stackMode">default</option>
    <option name="charting.chart.style">shiny</option>
    <option name="charting.layout.splitSeries">0</option>
    <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
    <option name="charting.legend.placement">right</option>
    <option name="refresh.display">progressbar</option>
  </chart>
</panel>

Re: Issue with the search base

anjibabu — Fri, 17 Apr 2020 00:01:37 GMT

<query>index=perfmon source="Perfmon:LogicalDisk" counter="% Free Space"</query>
<earliest>$TimeRangePkr.earliest$</earliest>
<latest>$TimeRangePkr.latest$</latest>
<refresh>5m</refresh>
<refreshType>delay</refreshType>


<query>index=perfmon sourcetype="Perfmon:Memory" counter="% Committed Bytes In Use"</query>
<earliest>$TimeRangePkr.earliest$</earliest>
<latest>$TimeRangePkr.latest$</latest>
<refresh>5m</refresh>
<refreshType>delay</refreshType>

Re: Issue with the search base

anjibabu — Fri, 17 Apr 2020 00:02:20 GMT

could you pls advise

Re: Issue with the search base

jpolvino — Fri, 17 Apr 2020 01:44:45 GMT

Might not hurt to physically join the base query in front of the panel's query along with the time tokens. Do away with the base search for now.

Also, consider a debug HTML panel to validate your tokens (remove leading spaces):
< row>
< panel>
< html>
< h1>Debug< /h1>
< p>earliest=$TimeRangePkr.earliest$<br/>latest=$TimeRangePkr.latest$< /p>
< /html>
< /panel>
< /row>

Re: Issue with the search base

anjibabu — Fri, 17 Apr 2020 02:09:12 GMT

Hi,

actually we are creating multiple panels in the same dashboard, so wanted to use the search base to avoid the performance issues.

search id query "index=perfmon sourcetype="Perfmon:Memory" counter="% Committed Bytes In Use"
search base query " | search host = DMOPWMD1PDDB0*" timechart perc90(Value) by host limit=0 span=1m"

it is not working:(

Re: Issue with the search base

jpolvino — Fri, 17 Apr 2020 12:17:25 GMT

When I run into this issue, here's what I do, and it might work for you.
1. Clone the dashboard.
2. Remove all panels except one that is giving you issues.
3. Hard code the full search and the time range into the panel's search. In other words, use the earliest and latest keywords in the search itself before the first pipe. Example: (main search elements) earliest=-1w@w latest=now | search host=.... Test it.
4. Add your time picker input, remove the earliest and latest from your search, and make the search use your time picker tokens. Test it.
5. Remove the front part of the working search to make your base search, use the time tokens in that search, and make your main search use the base. Test it.

Re: Issue with the search base

anjibabu — Mon, 20 Apr 2020 01:26:56 GMT

Thank you @jpolvino.

I have tried the above steps, but no luck, still same issue:(

Thanks,

Re: Issue with the search base

wyfwa4 — Thu, 30 Apr 2020 15:04:11 GMT

From my interpretation of your base search - the issue is that the base search is not transforming. This is a requirement for post-processed searches

https://docs.splunk.com/Documentation/Splunk/8.0.3/Viz/Savedsearches#Post-process_searches_2

If you take a simple search with stats/chart command and then run it in the standard search window, you will get the results you want. However if you split this in a form/dashboard and only have the initial search in the base search, you will not get any results from your post processing. You will need to add a stats command or similar to the base search to generate a table of results before this will work.

If I take one of your examples, the full query is as follows

index=perfmon source="Perfmon:LogicalDisk" counter="% Free Space" | search host = DMOPWMD1PDDB0* | eval FreeSpace =100-( Value ) | stats min(FreeSpace) as hostavg by host,instance | table host,instance,hostavg | chart min(hostavg) by host,instance

It looks like you have created the following base search - however this only returns raw events and not an table

index=perfmon source="Perfmon:LogicalDisk" counter="% Free Space"

I would split this up as follows

Base search

index=perfmon source="Perfmon:LogicalDisk" counter="% Free Space" |  eval FreeSpace =100-( Value ) | stats min(FreeSpace) as hostavg by host,instance

Post-processing search

|search host = DMOPWMD1PDDB0* |chart min(hostavg) by host,instance

In many cases, you need to create a temporary stats table in the base search, just to get this to work, even if you would not normally do this in an interactive search. If you base search cannot be easily combined into a single stats table, then you can create multiple base searches. I don't see the code you are using for the search ID's so just in case, it needs to be in this general format

  <search id="BaseSearchName1">
    <query>index=........</query>
    <earliest>-24h</earliest>
    <latest>now</latest>
  </search>

Re: Issue with the search base

DalJeanis — Thu, 30 Apr 2020 15:46:51 GMT

The answer is thoroughly covered by @wyfwa4. You should accept that one, and upvote this one.

Here's the tl;dr -

A base search always runs in fast mode, which optimizes out everything it doesn't need. It cannot see the post process searches, and pays no attention to them. If you don't tell the base search which fields to keep, the base search will throw them all away.

Add a table or fields command at the end of the base search to keep each field you want to use in any of the post-process searches.

 | fields host instance counter Value