Speed up simple dashboard with lookups

[subtrakt]

Hi!

I have a very simple dashboard with URLLoader, Tabs, and the Switcher modules that is simply using lookups to display administrative csv files (ex. contact info etc)

Problem 1.
Lookups in a dashboard take a few seconds to load when tab is clicked. Is that normal? Splunk is actually running a search and it takes a few seconds after the tab is clicked to load the content in the lookup...

I would think this should be almost instantaneous when the tab is clicked.
I experimented with SavedSearch and HiddenSavedSearch modules but for some reason the saved searches are not being found when i change usehistory=true so I have to use "auto" until i figure out why the saved searches are not coming up. My guess is, my scheduled saved searches in this case aren't really "searches" per se, because it is simply |inputlookup and scheduled saved searches don't work with |inputlookup only? I'm not even sure the Scheduled Saved Search would be faster than an inline for a lookup but It was fun experimenting.

Problem 2.
I have 3 tabs. The tabs load the underlying lookups successfully every time they are clicked. At first when the tab is clicked it does the search thing. I go to another tab it searches and pulls up the lookup succesfully. However, I go back to the first tab, it has the content instantaneously since it was already loaded before (which is great) but then clears, reloads and would be annoying to a user.

Now that i think about it: If i can load all lookups for all tabs with the dashboard initial load, and the tabs had everything loaded and ready to go, that would solve both problems 1 and 2. Any help would be greatly appreciated.

The goal is to have instantaneous content when the tabs are clicked.

Here's the code:

<module name="URLLoader" layoutPanel="navigationHeader" autoRun="True">
<param name="keepURLUpdated">True</param> 

<module name="Tabs" layoutPanel="panel_row2_col1" autoRun="True">
<param name="name">selectedTab</param>
<param name="staticTabs">
<list>
<param name="label">test1</param>
<param name="value">test1</param>
</list>

<list>
<param name="label">test2</param>
<param name="value">test2</param>
</list>

<list>
<param name="label">test3</param>
<param name="value">test3</param>
</list>

</param>



    <module name="Switcher" group=" ">
        <param name="selectedGroup">$selectedTab$</param>

<module name="NullModule" group="test1">
<module name="HiddenSavedSearch" layoutPanel="panel_row3_col1" autoRun="True">
    <param name="useHistory">auto</param>
    <param name="savedSearch">contacts</param>
    <module name="Table" layoutPanel="panel_row3_col1">
    <param name="count">1000</param>
</module>
</module>
</module>  

[sideview]

Agreed, it should be extremely fast for Splunk to dispatch an inputlookup search, have that search complete and have the client side modules pull down the results. Especially when the lookup is very small, the whole thing should be instantaneous. Unfortunately there's a bit of overhead in the search dispatching logic inside the Splunk Server itself, and this adds about a second or two.

The worst thing in this case is that when the next search is dispatched, the modules downstream clear themselves, and this is the "flashing". It's particularly annoying if it's literally taking away the content for a second, then reloading the same content.

Cooking up a testcase locally, with a combination of small lookups and large lookups, I'm surprised how long the searches take to "complete", even for the tiny lookups. the UI actually has to poll the search API several times before Splunkd gives the OK that the search is finished. So unless we can eliminate the actual search dispatches, there's not a lot that can be done.

Leaving out the Tabs module, the xml looks like this:

  <module name="Switcher" group=" ">
    <param name="selectedGroup">$selectedTab$</param>

    <module name="Search" group="test1">
      <param name="search">| inputlookup lookup1</param>
      <module name="Pager">
        <module name="Table" />
      </module>
    </module>   

    <module name="Search" group="test2">
      <param name="search">| inputlookup lookup2</param>
      <module name="Pager">
        <module name="Table" />
      </module>
    </module>  

    <module name="Search" group="test3">
      <param name="search">| inputlookup lookup3</param>
      <module name="Pager">
        <module name="Table" />
      </module>
    </module>
  </module>
</module>

So let's talk about why and where the searches are dispatching and redispatching. The answer is that the "dispatch points for the three searches are downstream from the Tabs modules". If your response to this is "huh?" then read the very important page in the Sideview Utils docs, "Introduction to the advanced xml".

To change it to make the push from the Tabs module not dispatch the search at all, we have to "move the dispatch point above the push". Yes this is a bit insane. Just read that page again. There's a reason why the Sideview Utils homepage tells you to read it a few times.

In a nutshell though, what's happening is that the Tabs module starts a cascading push downstream through the modules. Because of how the modules are arranged, the search(es) dont get dispatched until a point downstream from the Tabs.

But if we glom together all three inputlookup commands into one large search result, using the handy "append=t" option in the inputlookup command, then we can move this single giant Search above the Tabs, and then use three different PostProcess requests under the Tabs to pull out the right data.

<module name="Search">
  <param name="search">| inputlookup lookup1 | inputlookup lookup2 append=t | inputlookup lookup3 append=t </param>

  <module name="JobProgressIndicator" />

  <module name="Tabs" layoutPanel="panel_row1_col1" >
    <param name="name">selectedTab</param>
    <param name="staticTabs">
      <list>
        <param name="label">test1</param>
        <param name="value">test1</param>
      </list>
      <list>
        <param name="label">test2</param>
        <param name="value">test2</param>
      </list>
      <list>
        <param name="label">test3</param>
        <param name="value">test3</param>
      </list>
    </param> 


    <module name="Switcher" group=" ">
      <param name="selectedGroup">$selectedTab$</param>

      <module name="PostProcess" group="test1">
        <param name="search">search call_connected=*</param>
        <module name="Pager">
          <module name="Table" />
        </module>
      </module>   

      <module name="PostProcess" group="test2">
        <param name="search">search code=* NOT call_connected=*</param>
        <module name="Pager">
          <module name="Table" />
        </module>
      </module>  

      <module name="PostProcess" group="test3">
        <param name="search">search field=*</param>
        <module name="Pager">
          <module name="Table" />
        </module>
      </module>
    </module>
  </module>
</module>

Note that the Search is above the Tabs now. And note that there is a JobProgressIndicator just below, as a sibling to the Tabs. This means that when the Tabs module gets its data from upstream, the search will already have been kicked off. So subsequent clicks (and therefore pushes) at the Tabs module wont need to redispatch any search(es).

And note that one tricky part, is to figure out what search criteria in the PostProcess will tease apart the right lookups into the right tabs. the searches that I left in there are remnants of the three callmanager lookups I was using in my testcase, and you will have to find your own search terms.

And it is indeed much faster this way. There's hardly any noticeable flash.

[subtrakt]

Did u get a 404 error after letting the dashboard sit idle for a long period of time then attempt to change a tab? Besides that it is working great!

[subtrakt]

It's working and very quick now. I'm actually using hiddensavedsearch as the main search which seems to help on the initial load. One minor problem is the first tab post process has to be set to usehistory false or auto. Shouldn't it be able to find scheduled search? The other tabs set to true are working.

[subtrakt]

I am so thankful you did this. I was playing with append=t idea in my head the other day.

I will play around with this as soon as I get a chance!

[subtrakt · Accepted solution]

I should have known! lookups don't have _time for scheduled search to reference.

| eval Time=_time | eval _time=strptime(Time,"%Y-%m-%dT%T.%Q%:z")

Anyone know about how to get all the lookups loaded on the initial dashboard launch so everything is ready to go when tabs are switched?