topic Re: HiddenPostProcess pagination issue in Dashboards & Visualizations

HiddenPostProcess pagination issue

hbazan — Tue, 07 Sep 2010 22:00:43 GMT

Hello, I've created a view, with a textbox and a TimeRangePicker. I use those values on a HiddenSearch, and then several HiddenPostProcess, each one with a panel with a chart and a table. The problem I have is with the paginators for those tables. The number of pages seems to be computed for the HiddenSearch, not for each HiddenPostProcess, so every panel has the same number of pages, but only a few pages have actual content. I'll paste some xml code, please let me know if you find some mistake:

<view autoCancelInterval="90" isVisible="true"
objectMode="SimpleForm" onunloadCancelJobs="true"
template="dashboard.html">
  <label>Application Info</label>
  <module name="AccountBar" layoutPanel="appHeader" />
  <module name="AppBar" layoutPanel="navigationHeader" />
  <module name="Message" layoutPanel="messaging">
    <param name="filter">*</param>
    <param name="clearOnJobDispatch">False</param>
    <param name="maxSize">1</param>
  </module>
  <module name="Message" layoutPanel="messaging">
    <param name="filter">splunk.search.job</param>
    <param name="clearOnJobDispatch">True</param>
    <param name="maxSize">1</param>
  </module>
  <module name="TitleBar" layoutPanel="viewHeader">
    <param name="actionsMenuFilter">dashboard</param>
  </module>
  <module name="ExtendedFieldSearch" layoutPanel="viewHeader">
    <param name="replacementMap">
      <param name="arg">
        <param name="appId">
          <param name="value"></param>
        </param>
      </param>
    </param>
    <param name="field">App Id</param>
    <param name="q">Insert App GUID</param>
    <param name="intention">
      <param name="name">stringreplace</param>
      <param name="arg">
        <param name="appId">
          <param name="default">Insert App GUID</param>
          <param name="fillOnEmpty">True</param>
        </param>
      </param>
    </param>
    <module name="TimeRangePicker">
      <param name="searchWhenChanged">True</param>
      <module name="HiddenSearch" layoutPanel="panel_row2_col1"
      autoRun="True">
        <param name="search">
          <![CDATA[index="My_Index" $appId$]]>
</param>
        <module name="HiddenPostProcess"
        layoutPanel="panel_row1_col2" group="Info"
        autoRun="False">
          <param name="search">
            <![CDATA[search "some search value" | rex "field extraction (?<Action>.*) from here" | stats count by Action]]>
</param>
          <param name="groupLabel">Info</param>
          <module name="HiddenChartFormatter">
            <param name="chart">ratioBar</param>
            <param name="legend.placement">none</param>
            <module name="FlashChart">
              <param name="width">100%</param>
              <param name="height">50px</param>
            </module>
          </module>
        </module>
        <module name="HiddenPostProcess"
        layoutPanel="panel_row1_col2" group="Info"
        autoRun="False">
          <param name="search">
            <![CDATA[search "some search value" | rex "field extraction (?<Action>.*) from here among others" | table _time, Action, Field2 | sort +_time]]>
</param>
          <param name="groupLabel">Info</param>
          <module name="ViewstateAdapter">
            <module name="HiddenFieldPicker">
                            <module name="JobProgressIndicator"/>
              <param name="strictMode">True</param>
              <module name="Paginator">
                <param name="count">20</param>
                <param name="entityName">results</param>
                <module name="EnablePreview">
                  <param name="enable">True</param>
                  <param name="display">False</param>
                  <module name="SimpleResultsTable">
                    <param name="count">20</param>
                    <param name="drilldown">row</param>
                    <param name="allowTransformedFieldSelect">
                    True</param>
                    <param name="displayRowNumbers">true</param>
                    <module name="ConvertToDrilldownSearch">
                      <module name="ViewRedirector">
                        <param name="viewTarget">
                        flashtimeline</param>
                      </module>
                    </module>
                  </module>
                  <module name="ViewRedirectorLink">
                    <param name="viewTarget">flashtimeline</param>
                  </module>
                </module>
              </module>
            </module>
          </module>
        </module>
      </module>
    </module>
  </module>
</view>

Re: HiddenPostProcess pagination issue

sideview — Wed, 08 Sep 2010 06:18:09 GMT

Im afraid this is a known problem. The Paginator module is unable to account for the postProcess filtering. In other words it renders the page links as though there was no postProcess - as though you wanted to page through the unfiltered set (of events or results).

One could make a module that's a subclass of the Paginator, that knows to make a special check using a postProcess search of

<postProcessSearch> | stats count

Then it would take the 'count' field that came back and use that as it's "total".

I looked into implementing this for the Discover app and it's doable. However this would involve changing a bunch of the Paginator's logic to have that extra asynchronous step.

I/we need this in several places as well so Im sure someone will get around to writing this class before too long. Sorry that I/we havent already.

UPDATE: Sideview Utils provides a module called Pager which has the same behavior as Paginator except that it is not subject to this bug. If you download and install the Sideview Utils on your system you'll be able to switch over to using Pager instead of Paginator.

Re: HiddenPostProcess pagination issue

hbazan — Wed, 08 Sep 2010 20:52:09 GMT

Ok, it'll be great to have this fix done. Anyway, the view is usable, although I'll have to notice people about this issue.

Thanks for your reply.

Re: HiddenPostProcess pagination issue

Lowell — Thu, 25 Aug 2011 15:01:29 GMT

This is now possible with the Pager module in Sideview Utils (which was written by nick).

From the Splunk.Module.Pager docs:

Provides page links. ie "Prev 1 2 3 4 5 Next". This module behaves much like Splunk's Paginator module, except that Pager accounts for postProcess searches when they are present and automatically renders the correct number of page links. Make sure to read all documentation within the Sideview Utils app for many more key details.

Re: HiddenPostProcess pagination issue

rroberts — Thu, 25 Aug 2011 19:03:05 GMT

Thanks for this!

Re: HiddenPostProcess pagination issue

sideview — Fri, 26 Aug 2011 05:56:23 GMT

Yep. thanks Lowell for bumping up this question again. Pager lives.