Query fails from Dashboard, but succeeds from Search. Is this a licensing or permissions issue?

[mexscrabbler]

I'm using a Splunk application I developed myself 2 years ago. At the time, I had an Enterprise trial license. I saved off a number of configuration files (indexes, inputs, props, transforms and lookups). I also saved the XML source for 2 reports. I subsequentely reverted to the Free license and have not used the application since.

I have now reinstalled Splunk. I asked for a new Trial license, but Splunk remembered that on that machine I had reverted to Free, so it only allowed me to install a Free license.

I restored all the configuration files listed above manually, through the command line.
Now, when I try to run my Dashboard reports, the searches embedded in them never render, instead displaying "waiting for input". However, if I run any of the queries directly in Search, they all run ok, even when launching them from the Dashboard using the "open in Search" button.

Is this a licensing issue? A permissions issue?

Any input appreciated!

[mexscrabbler · Accepted solution]

Thank you @cmerriman and @nfilippi_splunk, I've solved my problem!

I did a hard stare and compare between the first six panels and the last two, and I discovered that I had the extra word "search" after the construct. Once I removed that from the first 6 panels, everything worked.

I'm not sure how that term got in there, since I imported the reports are based on xml queries I had saved previously.

In any case, thanks for your assistance!

[masonmorales]

"Waiting for input" usually indicates that there are tokens in your dashboard searches that do not have values yet.

[cmerriman]

do you have a submitButton="true" / "false" in the fieldset?
or autorun="true" / "false" in the fieldset?
or searchWhenChanged="true" / "false" in the inputs?
your dashboard might require a submit button or one of these options be set.

[nfilippi_splunk]

Is it possible to paste your XML source? Also, I assume that you are on 6.5, is that correct?

It's likely not a licensing or permission issue, otherwise you would see a different message than the "waiting for input..." message you are seeing. That message indicates that the dashboard search handler cannot resolve all dependencies, and is waiting to dispatch it.

[mexscrabbler]

Thanks for the feedback!

I've anonymized and am attaching the field chooser portion of my report and the first search panel.

I've tried each of cmerriman's suggestions without success.

Regarding nfillippi's suggestion, I've confirmed that the search handler is resolving all dependencies by selecting "Open in Search" from below the Dashboard panel, and confirming that the query expands properly. I've also tried a suggestion I've seen on the web stating that double "$" signs should be used around the field variable, but this doesn't make a difference.

Any further input appreciated!

Reports

<input type="dropdown" token="series" searchWhenChanged="true">
  <label>Pick an XXX by Name</label>
  <choice value="*">All</choice>
  <default>*</default>
  <search>
    <query>index=xxx sourcetype=XXX_HEADER_TYPE | stats count by XXX_NAME</query>
  </search>
  <fieldForLabel>XXX_NAME</fieldForLabel>
  <fieldForValue>XXX_NAME</fieldForValue>
</input>


<panel>
  <chart>
    <title>Distribution of XXX Categories  Real Time last 60 min</title>
    <search>
      <query>search autorun="true" index=xxx sourcetype=XXX_HEADER_TYPE | search XXX_NAME=$series$ | eval CATEGORY=case(U_XXX &gt;4, "5:Excellent", U_XXX &gt; 3, "4:Good", U_XXX &gt;=2, "3:Fair", U_XXX &gt;=1, "2:Poor", U_XXX &lt;11, "1:Bad") | timechart count(CATEGORY) by CATEGORY | fields _time, 5:Excellent, 4:Good, 3:Fair, 2:Poor, 1:Bad</query>
      <earliest>rt-1h</earliest>
      <latest>rt</latest>
    </search>
    <option name="charting.axisTitleX.visibility">visible</option>
    <option name="charting.axisTitleY.visibility">visible</option>
    <option name="charting.axisX.scale">linear</option>
    <option name="charting.axisY.scale">linear</option>
    <option name="charting.chart">column</option>
    <option name="charting.chart.nullValueMode">gaps</option>
    <option name="charting.chart.sliceCollapsingThreshold">0.01</option>
    <option name="charting.chart.stackMode">stacked100</option>
    <option name="charting.chart.style">shiny</option>
    <option name="charting.drilldown">all</option>
    <option name="charting.layout.splitSeries">0</option>
    <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
    <option name="charting.legend.placement">right</option>
    <option name="wrap">true</option>
    <option name="rowNumbers">false</option>
    <option name="dataOverlayMode">none</option>
    <option name="drilldown">cell</option>
  </chart>
</panel>

[cmerriman]

my suggestions need to be at the top. you have autorun="true" in the <query> try to take that out of the query.

    <fieldset submitButton="true" autorun="true">
        <input type="dropdown" token="series" searchWhenChanged="true">
...

[mexscrabbler]

Hi cmerriman,

Thanks for spotting that error. I've now retried submitButton="true" and autorun="true" in the fieldset section, both separate and together, and searchWhenChanged="true" in the input section, but it hasn't helped.

I did a bit more troubleshooting:
Only the last 2 out of 8 panels are returning results. Those panels use earliest=rt-5m and latest=rt. I've tried setting that in all the other panels, but it hasn't made any difference. I also tried setting those two panels to earlier=rt-1h, and they still work, so the difference must be in the query itself.

I noticed that the last two queries don't use tokens. In the first 6 queries I use the token "series", expressed as $series$, to store the value picked by the drop-down menu.

I went into the first panel and hard-coded the value of a token in the spot where I previously had "$series$" but I'm finding that the panel is still not refreshing (although it will launch the query if I click on the "Open in Search" button below the panel). The panel shows "No Results" instead of "waiting for data".

Any additional suggestions would be appreciated!!

[cmerriman]

does your input have any spaces or anything? you might need quotes around it?

maybe try something like

 | search XXX_NAME=$series|s$