All Apps and Add-ons

Splunk 6.x Dashboard Examples: Why am I getting "Invalid earliest_time" in the Chrome console opening the Pan and Zoom Chart Controls dashboard?

Builder

When I open the dashboard Pan and Zoom Chart Controls (simplechartrange_selection) from the app Splunk 6.x Dashboard Examples, I get the following error in the console of Chrome developer tools:

POST http://srv1:8000/en-US/splunkd/__raw/servicesNS/helge/simple_xml_examples/search/jobs?output_mode=js... 400 (Bad Request) mvc.js:6
[SPLUNKD] Invalid earliest_time. mvc.js:6

alt text
I am using the pan and zoom technique demonstrated on that dashboard in my app, where I get the same errors. This happens with Splunk 6.2.4 as well as Splunk 6.3 preview.

Is there some way to get rid of this error?

Update:

I monitored the failing searches with Fiddler and noticed, that earliest_time is set to the literal string $start$. This surely must be a bug!

alt text

Following is the full source code of my dashboard that causes the same errors (typically 6-7 identical errors when opening the dashboard):

<?xml version='1.0' encoding='utf-8'?>
<form script="main.js, analyze_timechart.js" stylesheet="main.css, analyze_timechart.css">

   <label>Analyze data over time</label>
   <description>This dashboard lets you interactively analyze any filtered subset of your data over time.</description>

   <fieldset autoRun="true">
      <input type="time" searchWhenChanged="true">
         <label>Time range:</label>
         <default>
            <earliest>-1h</earliest>
            <latest>now</latest>
         </default>
      </input>
      <input type="dropdown" token="FilterDatamodel" id="Input_FilterDatamodel">
         <choice value="Application_BrowserPerformanceChrome">Browser: Chrome</choice>
         <choice value="Application_BrowserPerformanceIE">Browser: IE</choice>
         <choice value="System_GpuUsage">Machine GPU</choice>
         <choice value="System_SystemPerformanceSummary">Machine performance</choice>
         <choice value="Process_NetworkTargetPerformance">Network performance</choice>
         <choice value="Application_OutlookPluginLoad">Outlook Plugin Load Performance</choice>
         <choice value="Process_ProcessDetail">Process Detail</choice>
         <choice value="Process_ProcessStartup">Process Startup</choice>
         <choice value="Session_SessionDetail_Session0">Session 0</choice>
         <choice value="Session_SessionDetail_Users">User Sessions</choice>
         <label>Data model:</label>
      </input>
      <input type="dropdown" token="FilterField" id="Input_FilterField">
         <label>Filter field:</label>
      </input>
      <input type="dropdown" token="FilterOperator" id="Input_FilterOperator">
         <choice value="is">is</choice>
         <choice value="isNot">is not</choice>
         <default>is</default>
         <label>Filter operator:</label>
         <prefix></prefix>
         <suffix></suffix>
      </input>
      <input type="text" token="FilterExpression" id="Input_FilterExpression">
         <label>Filter expression (e.g. "*name*"):</label>
         <default>*</default>
      </input>
   </fieldset>

   <search id="Search_Single">
      <query>
      </query>
   </search>

   <!-- Row 0 -->
   <row depends="$Single01Title$">
      <panel>
         <single depends="$Single01Title$">
            <title>$Single01Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single01Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single02Title$">
            <title>$Single02Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single02Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single03Title$">
            <title>$Single03Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single03Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single04Title$">
            <title>$Single04Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single04Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single05Title$">
            <title>$Single05Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single05Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>

         <html depends="$SingleRow2Header$">
            <h2 class="AboveSingle">$SingleRow2Header$</h2>
         </html>

         <single depends="$Single06Title$">
            <title>$Single06Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single06Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single07Title$">
            <title>$Single07Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single07Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single08Title$">
            <title>$Single08Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single08Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single09Title$">
            <title>$Single09Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single09Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single10Title$">
            <title>$Single10Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single10Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single11Title$">
            <title>$Single11Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single11Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
         <single depends="$Single12Title$">
            <title>$Single12Title$</title>
            <search base="Search_Single">
            </search>
            <option name="field">$Single12Field$</option>
            <option name="refresh.time.visible">false</option>
         <option name="height">50</option></single>
      </panel>
   </row>

   <!-- Row 1 -->
   <row>
      <panel>
         <title>Event and originating host count (try zooming in!)</title>
         <description>Zoom here to modify the time range for the chart below</description>
         <chart>
            <search>
               <query>
                  | pivot uberAgent $FilterDatamodel$
                     count($FilterDatamodel$) as Eventcount
                     dc(host) as Hostcount
                     splitrow
                        _time
                        period minute
                     filter $FilterField$ $FilterOperator$ "$FilterExpressionInternal$"
                  | timechart
                     sum(Eventcount) as "Event count"
                     sum(Hostcount) as "Host count"
                     `dynamic_span`
               </query>
            </search>
            <option name="height">150px</option>
            <option name="charting.chart">column</option>
            <option name="charting.legend.placement">none</option>
            <option name="charting.legend.masterLegend">null</option>
            <option name="charting.chart.overlayFields">"Host count"</option>
            <option name="charting.axisY2.enabled">true</option>
            <option name="charting.axisY2.fields">"Host count"</option>
            <option name="charting.axisY.includeZero">true</option>
            <option name="charting.axisY2.includeZero">true</option>
            <option name="charting.axisTitleX.visibility">collapsed</option>
            <option name="charting.axisTitleY.text">Events</option>
            <option name="charting.axisTitleY2.text">Hosts</option>
            <option name="charting.seriesColors">[0x324969, 0xFAC61D]</option>
            <selection>
               <set token="selection.earliest">$start$</set>
               <set token="selection.latest">$end$</set>
            </selection>
        </chart>
      </panel>
   </row>

   <search id="Search_FilterField">
      <query>
         | tstats
            dc("$FilterDatamodelRoot$.*")
            from
               datamodel=uberAgent.$FilterDatamodelRoot$
            where
               (nodename = $FilterDatamodelRoot$)
            prestats=true
            earliest=-15m latest=now
         | stats
            dedup_splitvals=t
            dc("$FilterDatamodelRoot$.*") as *
         | transpose
         | where column != "*"
      </query>
   </search>

   <!-- Row 2 -->
   <row depends="$selection.earliest$, $selection.latest$, $FilterDatamodelRoot$, $FilterLevel2$">
      <panel>
         <title>Compare two metrics over time</title>
         <input type="dropdown" token="Panel11Function1" id="Input_Panel11Function1">
            <choice value="avg">average</choice>
            <choice value="count">count</choice>
            <choice value="max">maximum</choice>
            <choice value="median">median</choice>
            <choice value="min">minimum</choice>
            <choice value="stdev">standard deviation</choice>
            <choice value="sum">sum</choice>
            <default>avg</default>
            <label>Function 1:</label>
         </input>
         <input type="dropdown" token="Panel11Field1" id="Input_Panel11Field1">
            <search base="Search_FilterField"></search>
            <fieldForLabel>column</fieldForLabel>
            <fieldForValue>column</fieldForValue>
            <label>Field 1:</label>
         </input>
         <input type="dropdown" token="Panel11Function2" id="Input_Panel11Function2">
            <choice value="avg">average</choice>
            <choice value="count">count</choice>
            <choice value="max">maximum</choice>
            <choice value="median">median</choice>
            <choice value="min">minimum</choice>
            <choice value="stdev">standard deviation</choice>
            <choice value="sum">sum</choice>
            <default>avg</default>
            <label>Function 2:</label>
         </input>
         <input type="dropdown" token="Panel11Field2" id="Input_Panel11Field2">
            <search base="Search_FilterField"></search>
            <fieldForLabel>column</fieldForLabel>
            <fieldForValue>column</fieldForValue>
            <label>Field 2:</label>
         </input>
         <chart depends="$selection.earliest$, $selection.latest$">
            <search>
               <query>
                  | pivot uberAgent $FilterDatamodel$
                     $Panel11Function1$($Panel11Field1$) as "$Panel11Function1$($Panel11Field1$)"
                     $Panel11Function2$($Panel11Field2$) as "$Panel11Function2$($Panel11Field2$)"
                     splitrow
                        _time
                     $FilterLevel2$
                     filter $FilterField$ $FilterOperator$ "$FilterExpressionInternal$"
               </query>
               <earliest>$selection.earliest$</earliest>
               <latest>$selection.latest$</latest>
            </search>
            <option name="height">350px</option>
            <option name="charting.chart">line</option>
            <option name="charting.legend.placement">bottom</option>
            <option name="charting.axisTitleX.visibility">collapsed</option>
            <option name="charting.chart.overlayFields">$Panel11Function2$($Panel11Field2$)</option>
            <option name="charting.axisY2.enabled">true</option>
            <option name="charting.axisY2.fields">$Panel11Function2$($Panel11Field2$)</option>
            <option name="charting.axisY.includeZero">true</option>
            <option name="charting.axisY2.includeZero">true</option>
         </chart>
      </panel>
   </row>

   <!-- Row 3 -->
   <row depends="$selection.earliest$, $selection.latest$">
      <panel id="Panel_31" depends="$Panel31Title$">
         <title>$Panel31Title$</title>
         <chart id="Chart_Panel31" depends="$selection.earliest$, $selection.latest$">
            <search id="Search_Panel31Chart">
               <query>
               </query>
               <earliest>$selection.earliest$</earliest>
               <latest>$selection.latest$</latest>
            </search>
            <option name="height">250px</option>
            <option name="charting.chart">line</option>
            <option name="charting.legend.placement">bottom</option>
            <option name="charting.axisTitleX.visibility">collapsed</option>
            <option name="charting.axisTitleY.visibility">collapsed</option>
            <option name="charting.axisY.includeZero">true</option>
         </chart>
      </panel>
   </row>

   <!-- Row 4 -->
   <row depends="$selection.earliest$, $selection.latest$">
      <panel id="Panel_41" depends="$Panel41Title$">
         <title>$Panel41Title$</title>
         <chart id="Chart_Panel41" depends="$selection.earliest$, $selection.latest$">
            <search id="Search_Panel41Chart">
               <query>
               </query>
               <earliest>$selection.earliest$</earliest>
               <latest>$selection.latest$</latest>
            </search>
            <option name="height">250px</option>
            <option name="charting.chart">column</option>
            <option name="charting.chart.stackMode">$Panel41ChartStackMode$</option>
            <option name="charting.legend.placement">bottom</option>
            <option name="charting.axisTitleX.visibility">collapsed</option>
            <option name="charting.axisTitleY.visibility">collapsed</option>
            <option name="charting.axisY.includeZero">true</option>
         </chart>
      </panel>
   </row>

   <!-- Row 5 -->
   <row depends="$selection.earliest$, $selection.latest$">
      <panel id="Panel_51" depends="$Panel51Title$">
         <title>$Panel51Title$</title>
         <chart id="Chart_Panel51" depends="$selection.earliest$, $selection.latest$">
            <search id="Search_Panel51Chart">
               <query>
               </query>
               <earliest>$selection.earliest$</earliest>
               <latest>$selection.latest$</latest>
            </search>
            <option name="height">250px</option>
            <option name="charting.chart">column</option>
            <option name="charting.chart.stackMode">$Panel51ChartStackMode$</option>
            <option name="charting.legend.placement">bottom</option>
            <option name="charting.axisTitleX.visibility">collapsed</option>
            <option name="charting.axisTitleY.visibility">collapsed</option>
            <option name="charting.axisY.includeZero">true</option>
         </chart>
      </panel>
   </row>

   <!-- Row 6 -->
   <row depends="$selection.earliest$, $selection.latest$">
      <panel id="Panel_61" depends="$Panel61Title$">
         <title>$Panel61Title$</title>
         <chart id="Chart_Panel61" depends="$selection.earliest$, $selection.latest$">
            <search id="Search_Panel61Chart">
               <query>
               </query>
               <earliest>$selection.earliest$</earliest>
               <latest>$selection.latest$</latest>
            </search>
            <option name="height">250px</option>
            <option name="charting.chart">column</option>
            <option name="charting.chart.stackMode">stacked</option>
            <option name="charting.legend.placement">bottom</option>
            <option name="charting.axisTitleX.visibility">collapsed</option>
            <option name="charting.axisTitleY.visibility">collapsed</option>
            <option name="charting.axisY.includeZero">true</option>
         </chart>
      </panel>
   </row>

   <!-- Row 7 -->
   <row depends="$selection.earliest$, $selection.latest$">
      <panel id="Panel_71" depends="$Panel71Title$">
         <title>$Panel71Title$</title>
         <chart id="Chart_Panel71" depends="$selection.earliest$, $selection.latest$">
            <search id="Search_Panel71Chart">
               <query>
               </query>
               <earliest>$selection.earliest$</earliest>
               <latest>$selection.latest$</latest>
            </search>
            <option name="height">250px</option>
            <option name="charting.chart">line</option>
            <option name="charting.legend.placement">bottom</option>
            <option name="charting.axisTitleX.visibility">collapsed</option>
            <option name="charting.axisTitleY.visibility">collapsed</option>
            <option name="charting.axisY.includeZero">true</option>
         </chart>
      </panel>
   </row>

   <!-- Row 8 -->
   <row depends="$selection.earliest$, $selection.latest$">
      <panel id="Panel_81" depends="$Panel81Title$">
         <title>$Panel81Title$</title>
         <table id="Table_Panel81" depends="$selection.earliest$, $selection.latest$">
            <search id="Search_Panel81Table">
               <query>
               </query>
               <earliest>$selection.earliest$</earliest>
               <latest>$selection.latest$</latest>
            </search>
            <option name="count">50</option>
            <option name="drilldown">none</option>
         </table>
      </panel>
   </row>
</form>

New Member

Has anyone found a solution to this problem? I am using 6.6.3 and this bug still seems to be around. I was able to get around the initial errors by using the below "selection" in the timeline charts in my dashboard but once a selection is made on one of the timeline charts then it seems to cancel and rerun the searches for the number of timeline charts on the dashboard that is using the below selection. i.e. If there are 3 charts using this selection then the dashboard will be refreshed 3 times.

<select>
     <eval token="form.TimeRange.earliest">if(isnotnull($start$),$start$,$form.TimeRange.earliest$)</eval>
     <eval token="form.TimeRange.latest">if(isnotnull($end$),$end$,$form.TimeRange.latest$)</eval>
</select>
0 Karma

Me too: I get the same "Invalid earliest_time" errors in Splunk 7.3.

In my case, the first viz in each dashboard is a timechart that I use to set $selection_earliest$ and $selection_latest$:

<selection>
    <set token="selection_earliest">$start$</set>
    <set token="selection_latest">$end$</set>
</selection>

The search for every other visualization in each dashboard uses:

<earliest>$selection_earliest$</earliest>
<latest>$selection_latest$</latest>

That "master" timechart is very useful for dynamically zooming and panning within the time range set by the time picker.

But... I get those ugly "Invalid earliest_time" errors.

I thought at first this was because I was missing:

depends="$selection_earliest$,$selection_latest$"

attributes. But, nope, even with that attribute on every applicable <row> and <search>, I still get those "Invalid earliest_time" errors.

0 Karma

I forgot to add: as @helge reports, I don't have to open the browser console to see these errors: they're right in your face, in the dashboard panels. It's disconcerting for users. If they report it, I can only shrug, say "it's Splunk behavior", and point them to this question.

0 Karma

Champion

Hello @helge,

I believe there are some bugs with the Splunk 6.x Dashboard Examples, and needs to be updated. As of 6.2 the Simple XML <earliestTime> and <latestTime> elements have been deprecated and replaced by <earliest> and <latest> elements. See the deprecated feature list. Many of the dashboards examples still use earliestTime. The app has not been updated since Oct. 28, 2014.

If you update the panels to use the new elements it should work. Be careful when upgrading as a few Simple XML tags have been deprecated in favor of new ones.

0 Karma

Builder

Thanks for your answer. Unfortunately that does not seem to have been the cause.
I had used earliestTime and latestTime in my app only for input type="time", the time range selector. With the individual charts I had already used earliest and latest.
I replaced the instances of earliestTime and latestTime with earliest and latest, respectively and even restarted Splunk - just to be on the safe side - but the errors did not go away.

0 Karma

Champion

Do you mind posting your xml?

0 Karma

Builder

No, I am happy to share my dashboard source code. It is a bit lengthy, but I added it to the question.
Please not that some logic is happening in JavaScript.

0 Karma

Champion

Are you creating or managing search jobs through your javascript?

0 Karma

Builder

Yes, I am managing some of the dashboard's searches from JavaScript. Here are two typical examples:

// Provoke a new data model field search
var searchFilterField = mvc.Components.getInstance("Search_FilterField");
searchFilterField.startSearch();

// Update search manager
var searchPanel31 = mvc.Components.getInstance("Search_Panel31Chart");
searchPanel31.settings.unset("search");
searchPanel31.settings.set("search", mvc.tokenSafe(searchString));
0 Karma

Champion

At what point do you experience the error and do you know which search instance is creating error? If you do i would use jconsole to print the object and inspect the search job properties.

0 Karma

Builder

I get the errors every time the dashboard is opened or refreshed. There is one error per chart.
While the charts are updating (takes a few seconds) they show "invalid earliest time". That message is then quickly replaced with the chart/data.

0 Karma

Builder

Updated the question with some interesting insights from Fiddler.

0 Karma

Champion

it looks as if you tokens are not being honored. Using the console
inspect the token model using the following.

console.log(mvc.Components.getInstance("default"));

This should let you view all tokens for that page.

0 Karma

Builder

The errors are temporary - they happen WHILE a dashboard is refreshed, but eventually data is displayed.
My suspicion is that there is a phase in which $start$ and $end$ are not defined. When they get defined later, the searches are repeated and things work.

0 Karma