topic Re: Real-time Sideview Utils dashboard using postprocessing not updating for module SimpleResultsTable, but updates for Table module in All Apps and Add-ons

Real-time Sideview Utils dashboard using postprocessing not updating for module SimpleResultsTable, but updates for Table module

aholzer — Mon, 28 Sep 2020 13:35:11 GMT

I have a dashboard that uses a real-time HiddenSearch to feed three separate panels via HiddenPostProcess modules. One of the three panels does not update upon new events coming in. When I run the full search (hidden search + post process) in the flashtime search window, the resulting table returns the correct values and updates as new events come in.

I was using the SimpleResultsTable for all three panels. I tested with multiple changes to the search to get the broken panel to start updating, but nothing was working. If, however, I leave the original search but change the SimpleResultTable module to the Table module instead, the panel updates as new events come in.

Is anyone aware of a bug in the SimpleResultsTable?

Here is the xml with the working module (I have only included two of the three panels, the panel_row2_col1 is the one that was causing the problems):

<module name="HiddenSearch" layoutPanel="panel_row1_col1" autoRun="True">
    <param name="search">
      `ems_header` POS_TURNOVER=* | `opening_pos_oid` | `macro_day` | `dedup_acct_ric` | `rename_fields` | `net_pos` | `pnl` | `open_net_pos`
    </param>
    <param name="earliest">rt-15m</param>
    <param name="latest">rt</param>

    <!-- Start: PNL panel -->
    <module name="HTML" layoutPanel="panel_row1_col1"> <!-- string header for panel -->
      <param name="html"><![CDATA[
        <h2>Cumulative PnL, Exposure, PnL and Turnover by Account</h2>
      ]]></param>
    </module>

    <module name="HiddenPostProcess" layoutPanel="panel_row1_col1">
      <param name="search">
        `showline(fields="CUM_PNL, EXP, PNL, TURN")` | stats sum(CUM_PNL) as CUM_PNL, sum(EXP) as EXP, sum(PNL) as PNL, sum(TURN) as TURN by ACCOUNT | `format_pnl_fields`
      </param>

      <module name="Pager">
        <param name="count">10</param>

        <module name="SimpleResultsTable"></module>
      </module>
    </module>
    <!-- End: PNL panel -->

    <!-- Start: Net Positions panel -->
    <module name="HTML" layoutPanel="panel_row2_col1"> <!-- string header for panel -->
      <param name="html"><![CDATA[
        <h2>Net Filled Positions and PnL by Account and Instrument</h2>
      ]]></param>
    </module>

    <module name="HiddenPostProcess" layoutPanel="panel_row2_col1">
      <param name="search"> 
        `showline(fields="NET_POS, CUM_PNL, EXP, PNL, TURN")` | table _time, ACCOUNT, RIC, NET_POS, CUM_PNL, EXP, PNL, TURN | sort +ACCOUNT, +RIC | `format_pnl_fields`
      </param>

      <module name="Pager">
        <param name="count">20</param>

        <module name="Table"/>
      </module>
    </module>
    <!-- End: Net Positions panel -->
...

Here is the xml with the non-updating module:

    <!-- Start: Net Positions panel -->
    <module name="HTML" layoutPanel="panel_row2_col1"> <!-- string header for panel -->
      <param name="html"><![CDATA[
        <h2>Net Filled Positions and PnL by Account and Instrument</h2>
      ]]></param>
    </module>

    <module name="HiddenPostProcess" layoutPanel="panel_row2_col1">
      <param name="search">
        `showline(fields="NET_POS, CUM_PNL, EXP, PNL, TURN")` | table _time, ACCOUNT, RIC, NET_POS, CUM_PNL, EXP, PNL, TURN | sort +ACCOUNT, +RIC | `format_pnl_fields`
      </param>

      <module name="Pager">
        <param name="count">20</param>

        <module name="SimpleResultsTable"></module>
      </module>
    </module>
    <!-- End: Net Positions panel -->

--- EDIT ---

I forgot to mention that I am using the latest version of sideview utils: v2.4.3

Re: Real-time Sideview Utils dashboard using postprocessing not updating for module SimpleResultsTable, but updates for Table module

sideview — Sat, 23 Mar 2013 02:08:14 GMT

It's hard to say without looking at the definitions of all your macros. Can you answer two questions -

1) if you were to run the base search only, in the flashtimeline view, would it render in the events list by default, or in the table view by default? I'm curious as to whether the base search is considered a transforming search.

2) for completeness it would be good to know what commands were inthe format_pnl_fields and show_line macros too.

My suspicion is that the complex logic around SimpleResultsTable's "entityName" param is leading to the bug.

Re: Real-time Sideview Utils dashboard using postprocessing not updating for module SimpleResultsTable, but updates for Table module

sideview — Sat, 23 Mar 2013 02:11:10 GMT

By the way, do you need Splunk's SimpleResultsTable module? the Table module is after all designed to replace it ultimately.

Re: Real-time Sideview Utils dashboard using postprocessing not updating for module SimpleResultsTable, but updates for Table module

aholzer — Mon, 28 Sep 2020 13:35:56 GMT

Here are your answers:
1) The base search displays in the event view
2) Here are the macro definitions, although the same macros work in other panels to the correct effect:
- format_pnl_fields uses the "fieldformat" command on a couple of the fields, all fields are formatted the same: fieldformat TURN=tostring(TURN,"commas") | ...
- showline(1): addtotals fieldname=showline $fields$ | where showline!=0

Regarding your other comment, I don't need the SimpleResultsTable, but I wanted it for formatting consistency. I will probably just change all panels to Table now for consistency.

Thanks

Re: Real-time Sideview Utils dashboard using postprocessing not updating for module SimpleResultsTable, but updates for Table module

sideview — Sat, 08 Jun 2013 06:43:55 GMT

I'm not sure if you ever figured out what was causing this, but it seemed the easy answer was to just use the Sideview Table module instead of Splunk's SimpleResultsTable module. So for the sake of the question having some answer I thought I'd write this up. =/

Re: Real-time Sideview Utils dashboard using postprocessing not updating for module SimpleResultsTable, but updates for Table module

aholzer — Sun, 09 Jun 2013 13:26:53 GMT

That did end up being the answer 🙂