Single-Value rangecolor issue when missing values

[pclewis]

The basic issue I faced was a dashboard with prominent single-value visualisation what was to display a count of exceptions.  The users wanted 0 exceptions to be "good" color and a range of colors after that.

To demonstrate, here is a simple test dashboard making use fo the excellent features of single-value viz.

<form>
  <label>test single value viz</label>
  <fieldset submitButton="false">
    <input type="text" token="limit">
      <label>limit</label>
      <default>2</default>
      <initialValue>2</initialValue>
    </input>
  </fieldset>
  <row>
    <panel>
      <single>
        <search>
          <query>| gentimes start=1/25/2022 end=1/26/2022 increment=1h 
| eval count=random()%$limit$ 
| eval _time=starttime
| table _time count
| timechart span=6h sum(count) as count</query>
          <earliest>-1h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="colorBy">value</option>
        <option name="colorMode">block</option>
        <option name="drilldown">none</option>
        <option name="numberPrecision">0</option>
        <option name="rangeColors">["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41"]</option>
        <option name="rangeValues">[0,30,70,100]</option>
        <option name="showSparkline">1</option>
        <option name="showTrendIndicator">1</option>
        <option name="trellis.enabled">0</option>
        <option name="trellis.scales.shared">1</option>
        <option name="trellis.size">medium</option>
        <option name="trendColorInterpretation">standard</option>
        <option name="trendDisplayMode">absolute</option>
        <option name="unitPosition">after</option>
        <option name="useColors">1</option>
        <option name="useThousandSeparators">1</option>
      </single>
    </panel>
  </row>
</form>

Default limit of 2 will result in a viz showing a lovely blue background and some values and trendline depending on the random data generated.
limit 20 will produce most likely an orange background
limit 200 a red background

All this is expected and in accordance with the default viz that was produced by using the "Save As Dashboard Panel" option from the base window.

A limit of 1 - which results in all data values of 0 gives the green background.  This is still expected.

Where I struggle is the limit of 0 (or less) which will give no data as number % 0 is undefined.  The data for such a search has no values in the count column.   

So what to do?  The single value viz has decided that null values are nearer max value than min value which makes sense if you use dafault colors because max value is colored red.  But if in your situation your low values are more abberent and you consider null values are abberations you'd want to have the nulls colored like your min value.  Also strange though si the value on the chart shows 0, even if all the values in the data set are null.  Suddenly null became 0 and not undefined, and thus 0 is treated as higher than max instead of lower than min.  I find this to be a mistake - either it's treated as 0 so color it as 0 and show it as 0 or it's treated as null so colour it as null and show it as null (or undefinied or something other than 0)

The only workaround I could find (without looking at css chnages) is a bit ugly and may not suit all situations.
I cludge the upper limit to some value "higher than I could ever reach" (famous last words) and stick the colour I want to display for no data there.

        <option name="rangeColors">["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41","0x53a051"]</option>
        <option name="rangeValues">[0,30,70,100,100000000]</option>

In the real world situation I had, zero values was considered good, and no data at all is also good so the quick fix of the viz as above was enough to allow users to visualise the date.

A better solution is to change the base search to something that always returned a 0  rather than null or add a line after the timechart to force nulls to an acceptable value.   I like the latter as it's far more clear what's going on.

| eval count=coalesce(count ,0)

When no data at all is returned by base search (as happened in my real world case) it can be handled the normal way with hidden panel to display when no data returned.  Side note on this: I usually have a panel that displays when there is no data for the base search but there is some data in the index/sourcetype and a different panel when there is no data at all.  This is because on rare occasions you may have a problem with a forwarder or any number of other reasons resulting in events taking longer than expected to appear in an index.  Letting user know this is the case rather than assuming "all's good" is better in my view.

In real world data it's ugly to manipulate source into vizualisation just to make it look right.  Sometimes we have to, but here I think the single-value vizualisation needs an option to let the user decide how to display missing or null values.