Realtime search in dashboard slow compared to realtime in flashtimeline

[KarunK]

Hi All,

I have a realtime search to find TPS in a dashboard. But the search in dashboard runs ten times slower than the same search run on search window. Couldn't figure out why. Also some times the data gets truncated as well.

Could anyone help ?

Update 29th April : I think the backfill is not working. How can the realtime backfill be enabled ?

Thanks

Regards

KK

search

index="router" $service$ hostname="$hostname$" | timechart span=1s count by hostname | timechart span=1min max(*)  as *

Advanced XML

<?xml version="1.0" encoding="UTF-8"?>
<view isPersistable="true" isSticky="false" isVisible="true" objectMode="viewconf" onunloadCancelJobs="true" stylesheet="application.css" template="dashboard.html">
   <label>Device</label>
   <module name="SideviewUtils" layoutPanel="messaging" />
   <module name="AccountBar" layoutPanel="messaging" />
   <module name="AppBar" layoutPanel="navigationHeader" />
   <module name="Message" layoutPanel="viewHeader">
      <param name="filter">*</param>
      <param name="clearOnJobDispatch">False</param>
      <param name="maxSize">2</param>
   </module>
   <module name="Message" layoutPanel="viewHeader">
      <param name="filter">splunk.search.*</param>
      <param name="clearOnJobDispatch">True</param>
      <param name="maxSize">1</param>
   </module>
   <module name="HiddenSearch" layoutPanel="panel_row1_col1" autoRun="True">
      <param name="search">|inputlookup address.csv</param>
      <module name="Pulldown">
         <param name="float">left</param>
         <param name="searchFieldsToDisplay">
            <list>
               <param name="value">hostname</param>
               <param name="label">hostname</param>
            </list>
         </param>
         <param name="name">hostname</param>
         <param name="postProcess">| inputlookup address | dedup hostname | table hostname | sort hostname</param>
         <param name="label">Device</param>
         <module name="Pulldown" layoutPanel="panel_row1_col1">
            <param name="searchFieldsToDisplay">
               <list>
                  <param name="value">service</param>
                  <param name="label">Delivery Service</param>
               </list>
            </param>
            <param name="outerTemplate">( $value$ )</param>
            <param name="label">Delivery Service</param>
            <param name="separator">+OR+</param>
            <param name="size">3</param>
            <param name="postProcess">| inputlookup service | dedup service | table service | sort service</param>
            <param name="name">service</param>
            <param name="template">$value$</param>
            <param name="float">left</param>
                          <module name="SubmitButton">
                  <param name="label">Search</param>

            <module name="Search" layoutPanel="panel_row2_col1" autoRun="True">
               <param name="search">index="router" $service$ hostname="$hostname$" | timechart span=1s count by hostname | timechart span=1min max(*)  as *</param>
               <param name="earliest">rt-1h</param>
               <param name="latest">rt</param>
               <module name="HTML" layoutPanel="panel_row2_col1">
                  <param name="html">&lt;pre&gt;
searchExpression : index="router" &lt;b&gt;$service$ hostname="$hostname$" &lt;/b&gt; | timechart span=1s count by hostname | timechart span=1min max(*)  as *
  &lt;/pre&gt;</param>
               </module>
               <module name="JobProgressIndicator" />
               <module name="JobStatus">    
            <param name="showCreateMenu">false</param>
             <param name="showSaveMenu">false</param> 
             </module>

               <module name="EnablePreview">
                  <param name="enable">True</param>
                  <param name="display">False</param>
                  <module name="HiddenChartFormatter" layoutPanel="panel_row2_col1" group="Real Time Service Router Peak TPS ( 1 hour window )">
                     <param name="groupLabel">Real Time TPS</param>
                     <param name="charting.chart">area</param>
                     <param name="primaryAxisTitle.text">Time</param>
                     <param name="secondaryAxisTitle.text">TPS</param>
                     <module name="FlashChart">
                     <param name="height">350px</param>
                        <module name="ConvertToDrilldownSearch">
                           <module name="ViewRedirector">
                              <param name="viewTarget">flashtimeline</param>
                           </module>
                        </module>
                     </module>
                  </module>
               </module>
            </module>
         </module>
      </module>
      </module>   
      </module>      
</view>

A simple xml dashboard was as fast as the flash-timeline one. Its only Advanced xml dashboard is slow.

<?xml version='1.0' encoding='utf-8'?>
<dashboard>
  <label>rrr</label>
  <row>
    <chart>
      <searchName>testinggggggggggggg</searchName>
      <title>testinggggggggggggg</title>
      <option name="charting.chart">area</option>
    </chart>
  </row>
</dashboard>

[KarunK · Accepted solution]

Idendified as a bug in Sideview/Splunk Core Engine.

Please refer the following link for workaround.
http://splunk-base.splunk.com/answers/85455/backfill-not-working-for-a-realtime-dashboard

[sideview]

The bug was in Splunk but deep enough that it may have only ever affected Sideview Utils. The relevant Splunk code has been patched by Sideview Utils as of a version or two ago so go ahead and update to latest SVU and the problem will completely go away.