Splunk Search
Highlighted

Real-time table in dashboard is not real-time, until you sort it

Motivator

Hi there,

I have a real-time table in one of my dashboards that doesn't update when you first load the page. If you do something with the table - e.g. click one of the fields to sort, it becomes real-time and starts updating in real-time. I'd like to have it be in real-time when you open the dashboard, and would rather not have to instruct everyone who uses the dashboard that you have to sort by a field in order to get it working.

Tags (2)
0 Karma
Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Motivator

My code, in case you'd like to take a look:

<form>
  <label>title</label>

  <row>
    <panel>
      <title>table title</title>
      <table>
        <search>
          <!-- comment -->
          <query>
index=myindex sender_domain != "abc.com" event_id="DELIVER" | lookup "mycsv.csv" sender | where isnull(precedent) |
mvexpand recipient | dedup sender message_subject recipient | where recipient_domain=="abc.com" | eventstats count as recipient_count by sender,message_subect | 
dedup sender,message_subject | sort - time | table sender message_subject recipient_count _time
          </query>
          <earliest>rt-5m</earliest>
          <latest>rt</latest>
        </search>
        <option name="wrap">true</option>
        <option name="rowNumbers">false</option>
        <option name="drilldown">cell</option>
        <option name="dataOverlayMode">none</option>
        <option name="count">25</option>
      </table>
    </panel>
  </row>

</form>
0 Karma
Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Path Finder

Are the other tables in the dashboard updating correctly? If so, have you tried creating a single dashboard with the above code to see if it runs correctly?

0 Karma
Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Motivator

Not using any other real-time tables, but yes right now this code is on a dashboard by itself.

0 Karma
Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Splunk Employee
Splunk Employee

Having a look at http://docs.splunk.com/Documentation/Splunk/7.1.3/Search/Specifyrealtimewindowsinyoursearch shows the setting default_backfill in limits.conf might have an impact on this behaviour.

Also if you leave the dashboard alone for 5 minutes does it then start real-time updating?

Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Motivator

Looks like we're closing in on the answer. However, default_backfill is already set to true (and experimentally setting it to false in etc/system/local/limits.conf leads to the same behavior). I wonder if another setting in this file is the solution?

And yes, after roughly 5 minutes it does start updating, which makes sense in accordance with:

For example, if your sliding window is 5 minutes, you will not start to see data until after the first 5 minutes have passed.

The solution the documentation provides is

You can override this behavior so that Splunk software backfills the initial window with historical data before running in the normal real-time search mode.

but default_backfill is already set to true. So I'm not sure how to proceed.

0 Karma
Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Splunk Employee
Splunk Employee

Hi,

Double check the default_backfill setting with btool http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Usebtooltotroubleshootconfigurati... if it says true then create a self contained app (with problematic dashboard with self contained SPL) and send to support. ( so it is easier for support to remotely test this as well ). It feels like this setting isn't being honoured.

Thanks

Mark

Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Motivator

Thanks!

./splunk cmd btool limits list

does list it as true, and

./splunk cmd btool --app=myapp limits list

returns nothing, which makes sense, since there is no local or default limits.conf file in my app directory. So it should default to the system limits.conf, which is indeed set to true according to btool.

I guess the next step is to contact support.

0 Karma
Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Motivator

Also, setting default_backfill to false in etc/system/local/limits.conf and running btool says that the value has been set to false. But then the dashboard still backfills 5 minutes (and still doesn't update until you sort it)

So.... it seems the dashboard for some reason knows to backfill 5 minutes, without having read etc/system/local/limits.conf... but when it backfills 5 minutes, it just doesn't update in real-time until you click sort.

0 Karma
Highlighted

Re: Real-time table in dashboard is not real-time, until you sort it

Motivator

This was fixed after I upgraded from 6.2

View solution in original post

0 Karma