All Apps and Add-ons

Use Gate to generate a temporary variable from a secondary search and use it over the primary search

alvaromari83
Path Finder

Hello everyone,

I'm having some issues with the following scenario.

I have a panel where I want to display a table with the following postprocessed results of a "search A". Search A generates a complete result set and postProcess searches A1, A2, A3 and A4 generate filtered results from search A:

  • A1, A2 and A3 will be displayed and navigated with a Tabs module, fast and easy, with the following, typical structure:


    SEARCH A QUERY


    tab_selector
    with tab values A1, A2 and A3

    <module name="Switcher">
      <param name="selectedGroup">$tab_selector$</param>
      <module name="PostProcess" group="A1">
        <param name="search">POSTPROCESS A1 QUERY where(condition A1) | ... and display results</param>
      </module>      
      <module name="PostProcess" group="A2">
        <param name="search">POSTPROCESS A2 QUERY where(condition A2) | ... and display results</param>
      </module>      
      <module name="PostProcess" group="A3">
        <param name="search">POSTPROCESS A3 QUERY where(condition A3) | ... and display results</param>
      </module>
    </module>
    


Then there is a second search, search B, which displays a timechart with a JSChart and, when clicked, generate a clicked.value variable I want to use to filter the last PostProcess (A4). Something like:

where(conditionA4=$clicked.value$) | ... and display results

When the search B timechart is clicked, the PostProcess A4 would ideally show the results in the panel automatically picking the A4 Tab selected. This is, however, not feasible as far as i know. So, I just override all the tabs module and generate a different panel for A4.

The goal is: having a search A, generate 4 postprocess searchs. 3 of them (A1,A2,A3) would be switchable and selectable through a Tabs module. The 4th one, A4, would be selectable through a timechart graph drilldown, when clicking a value ($clicked.value$ variable inside JSChart)

This is the pseudocode I've come to design:

<module name="ValueSetter">
  <param name="name">is_chartB_clicked</param>
  <param name="value">False</param>

 <module name="Search">
   <param name="search">SEARCH A QUERY</param>

   <module name="Gate">
     <param name="id">teleporter</param>

   <module name="Switcher">
     <param name="selectedGroup">$is_chartB_clicked$</param>

   <module name="Tabs" group="False" layoutPanel="panel_row1_col1">
     <param name="name">tab_selector</param>
     <param name="staticValues">with tab values A1, A2 and A3</param>

     <module name="Switcher">
       <param name="selectedGroup">$tab_selector$</param>
       <module name="PostProcess" group="A1">
         <param name="search">POSTPROCESS A1 QUERY where(condition A1) | ... and display results</param>
       </module>      
       <module name="PostProcess" group="A2">
         <param name="search">POSTPROCESS A2 QUERY where(condition A2) | ... and display results</param>
       </module>      
       <module name="PostProcess" group="A3">
         <param name="search">POSTPROCESS A3 QUERY where(condition A3) | ... and display results</param>
       </module>
     </module>
   </module>

   <module name="PostProcess" group="True" layoutPanel="panel_row1_col1">
         <param name="search">POSTPROCESS A4 QUERY where(condition A4=$clickedValueFromChartB$) | ... and display results</param>
    </module>
    </module>
    </module>
 </module>
</module>

...

<module name="Search" layoutPanel="panel_row2_col1">
    <param name="search">SEARCH B QUERY</param>
    <module name="JSChart">

     <module name="ValueSetter">
       <param name="name">is_chartB_clicked</param>
       <param name="value">True</param>


      <module name="ValueSetter">
        <param name="name">clickedValueFromChartB</param>
        <param name="value">$clicked.value$</param>

        <module name="Gate">
          <param name="to">teleporter</param>
        </module>
      </module>
    </module> 
  </module>
</module>

This is not working because the PostProcess A4 coming from search B to retrieve losses the search A results after the Gate teleport. However, i need search B to retrieve the click.value from the timechart to make the postProcess A4 filtering...

is there anyway to preserve on standby previous search results (search A results), to use later?? Any other idea to implement my main goal (render filtered results from different sources (Tabs and JSChart click drilldown in the same target panel) in a different way?

Thank you a lot.

Alvaro

1 Solution

sideview
SplunkTrust
SplunkTrust

I would back away from the Gate module approach. A good rule of thumb for Gate, is that if you think you need it, you've probably overlooked someway to slightly rearrange the page and make the hard problems go away.

To restate your requirements: Your page has an "interactive timechart" bit, and then a bunch of stuff in tabs. The tabs are each a postprocessed slice of search results from Search A. Also, when the user clicks the timechart, you want to also leverage Search A's results, and postprocess them partly using arguments from the timechart's drilldown click.

Let's for the moment let go of the idea that the timechart drilldown stuff should somehow appear in the tabs, and just say they have to appear somewhere, perhaps as a table or chart right under the timechart itself. This makes things a lot easier. We simply have two panels that each want to use the same search results in arbitrary ways. As with pretty much everything in the splunk UI, in order for two things to have access to the same data, that data must be upstream from both things.

So, let's rearrange the page to have our Search A actually dispatch at a point that is upstream from both panels. To prevent our timechart search from obliterating all evidence of Search A, let's use the addinfo command and the ResultsValueSetter module to save off the search id of Search A, as $info_sid$. Then we can use things like | loadjob $info_sid$ wherever we like in all downstream config.

Here is a working example illustrating your use case, but reworked to run against generic index=_internal data.

<module name="Search" layoutPanel="panel_row1_col1" autoRun="True">
  <param name="search"><![CDATA[
    index=_internal sourcetype=splunkd component=metrics | bin _time span=5min | stats sum(kb) as kb count by _time group series | addinfo
  ]]></param>
  <param name="earliest">-4h@h</param>

  <module name="ResultsValueSetter">
    <param name="fields">info_sid</param>

    <module name="HTML">
      <param name="html"><![CDATA[
        <h3>Random timechart search.</h3> Note that the drilldown here will leverage sid of the upstream search.
      ]]></param>
    </module>
    <module name="Search" >
      <param name="search"><![CDATA[
        index=_internal sourcetype=splunkd component=metrics group=per_sourcetype_thruput | timechart span=1h count by series
      ]]></param>

      <module name="ValueSetter">
        <param name="arg.charting.chart">column</param>
        <param name="arg.charting.chart.stackMode">stacked</param>

        <module name="JSChart">

          <module name="Search">
           <!-- one nasty thing here is that loadjob doesn't pay attention to earliest/latest api args,  
                  so we have to manually insert them into the search language. normally you should NOT 
                  worry about earliest/latest in the language, but here we have to -->
            <param name="search">| loadjob $info_sid$ | search _time>=$search.timeRange.earliest$ _time<$search.timeRange.latest$ $click.searchTerms$ | timechart sum(kb) by series</param>

            <module name="HTML">
              <param name="html"><![CDATA[
                <h3>Showing KB indexed for sourcetype=$click.name2$ $search.timeRange.label$</h3>
              ]]></param>
            </module>
            <module name="JSChart" />
          </module>
        </module>
      </module>
    </module>

    <module name="Tabs" layoutPanel="panel_row2_col1">
      <param name="name">tab_selector</param>
      <param name="staticTabs">
        <list>
          <param name="value">A1</param>
          <param name="label">sources</param>
        </list>
        <list>
          <param name="value">A2</param>
          <param name="label">sourcetypes</param>
        </list>
      </param>

      <module name="Switcher" group=" ">
        <param name="selectedGroup">$tab_selector$</param>

        <module name="PostProcess" group="A1">
          <param name="search">search group=per_source_thruput | stats sum(count) as count by series</param>

          <module name="ValueSetter">
            <param name="arg.charting.chart">bar</param>

            <module name="JSChart" />
          </module>
        </module>

        <module name="PostProcess" group="A2">
          <param name="search">search group=per_sourcetype_thruput | stats sum(count) as count by series</param>

          <module name="ValueSetter">
            <param name="arg.charting.chart">bar</param>

            <module name="JSChart" />
          </module>
        </module>
      </module>
    </module>
  </module>
</module>

And yes, there's probably a way to do this using the Gate module, but Gate offers a kind of black magic that is best saved for when there really is no other option.

View solution in original post

sideview
SplunkTrust
SplunkTrust

I would back away from the Gate module approach. A good rule of thumb for Gate, is that if you think you need it, you've probably overlooked someway to slightly rearrange the page and make the hard problems go away.

To restate your requirements: Your page has an "interactive timechart" bit, and then a bunch of stuff in tabs. The tabs are each a postprocessed slice of search results from Search A. Also, when the user clicks the timechart, you want to also leverage Search A's results, and postprocess them partly using arguments from the timechart's drilldown click.

Let's for the moment let go of the idea that the timechart drilldown stuff should somehow appear in the tabs, and just say they have to appear somewhere, perhaps as a table or chart right under the timechart itself. This makes things a lot easier. We simply have two panels that each want to use the same search results in arbitrary ways. As with pretty much everything in the splunk UI, in order for two things to have access to the same data, that data must be upstream from both things.

So, let's rearrange the page to have our Search A actually dispatch at a point that is upstream from both panels. To prevent our timechart search from obliterating all evidence of Search A, let's use the addinfo command and the ResultsValueSetter module to save off the search id of Search A, as $info_sid$. Then we can use things like | loadjob $info_sid$ wherever we like in all downstream config.

Here is a working example illustrating your use case, but reworked to run against generic index=_internal data.

<module name="Search" layoutPanel="panel_row1_col1" autoRun="True">
  <param name="search"><![CDATA[
    index=_internal sourcetype=splunkd component=metrics | bin _time span=5min | stats sum(kb) as kb count by _time group series | addinfo
  ]]></param>
  <param name="earliest">-4h@h</param>

  <module name="ResultsValueSetter">
    <param name="fields">info_sid</param>

    <module name="HTML">
      <param name="html"><![CDATA[
        <h3>Random timechart search.</h3> Note that the drilldown here will leverage sid of the upstream search.
      ]]></param>
    </module>
    <module name="Search" >
      <param name="search"><![CDATA[
        index=_internal sourcetype=splunkd component=metrics group=per_sourcetype_thruput | timechart span=1h count by series
      ]]></param>

      <module name="ValueSetter">
        <param name="arg.charting.chart">column</param>
        <param name="arg.charting.chart.stackMode">stacked</param>

        <module name="JSChart">

          <module name="Search">
           <!-- one nasty thing here is that loadjob doesn't pay attention to earliest/latest api args,  
                  so we have to manually insert them into the search language. normally you should NOT 
                  worry about earliest/latest in the language, but here we have to -->
            <param name="search">| loadjob $info_sid$ | search _time>=$search.timeRange.earliest$ _time<$search.timeRange.latest$ $click.searchTerms$ | timechart sum(kb) by series</param>

            <module name="HTML">
              <param name="html"><![CDATA[
                <h3>Showing KB indexed for sourcetype=$click.name2$ $search.timeRange.label$</h3>
              ]]></param>
            </module>
            <module name="JSChart" />
          </module>
        </module>
      </module>
    </module>

    <module name="Tabs" layoutPanel="panel_row2_col1">
      <param name="name">tab_selector</param>
      <param name="staticTabs">
        <list>
          <param name="value">A1</param>
          <param name="label">sources</param>
        </list>
        <list>
          <param name="value">A2</param>
          <param name="label">sourcetypes</param>
        </list>
      </param>

      <module name="Switcher" group=" ">
        <param name="selectedGroup">$tab_selector$</param>

        <module name="PostProcess" group="A1">
          <param name="search">search group=per_source_thruput | stats sum(count) as count by series</param>

          <module name="ValueSetter">
            <param name="arg.charting.chart">bar</param>

            <module name="JSChart" />
          </module>
        </module>

        <module name="PostProcess" group="A2">
          <param name="search">search group=per_sourcetype_thruput | stats sum(count) as count by series</param>

          <module name="ValueSetter">
            <param name="arg.charting.chart">bar</param>

            <module name="JSChart" />
          </module>
        </module>
      </module>
    </module>
  </module>
</module>

And yes, there's probably a way to do this using the Gate module, but Gate offers a kind of black magic that is best saved for when there really is no other option.

alvaromari83
Path Finder

Nice! Many thanks! addinfo and loadjob covers my issue. However, I'm still wondering how to make both actions (tab clicks and timechart drilldown click) to be rendered in the same panel (replacing the visualizations, not "getting added below"), without using Gate modules...

Thank you once again.

Alvaro

0 Karma

sideview
SplunkTrust
SplunkTrust

Well, the JSChart module lacks the Table module's "default" param options, meaning that barring custom JS patching, modules downstream from JSCharts will be hidden by default. So that rules out actually putting the set of Tabs downstream from the JSChart.

And this does indeed take us back to the Gate module. 😃 But at least with the loadjob trick we have more degrees of freedom. I would explore having a Gate downstream from the JSChart and the other Gate just upstream from the Tabs module. And also bear in mind that if a Tabs module receives a $foo$ token from upstream, whose name (ie foo) matches it's own "name" exactly, and whose value matches one of it's values, it will switch to that Tab then and there.... So you could put an argument into the JSChart, send it across the Gate, and that should take care of the autoswitching part. The part of actually using $click.searchTerms$ should be fairly obvious. And my last piece of advice is try not to worry about passing the timerange explicitly - most likely it'll just get passed through the Gate when relevant without any extra effort.

0 Karma