All Apps and Add-ons

Sideview Utils PostProcess and local csv inputlookup too slow!

Path Finder

Hello all!

I'm implementing a search panel with 2 sideview pulldowns. First one is just made of 3 static options, that serve as arguments in the nested second pulldown module, which queries its values from a csv lookup input file, using a PostProcess module. The query for this inputlookup is:

| inputlookup file.csv | where fuente="source1" | fields nombre valor

This query is running fast as hell in the search app, as expected, as the csv itself is just a few rows with the following format:

fuente , nombre , valor
source1 , Matricula , cot_carplate
source1 , Nombre , cot_nombre
source1 , Documentos , cot_id
soruce2 , Numero pol , pol_pol

The view XML is the following:

<!-- First pulldown list, static values for sourcetypes -->

fuente
Fuente de datos:
left


Cotizaciones
source1


Polizas
source2


Siniestros
source3


Busqueda libre


<!-- Second pulldown list to select the search parameters to use. It depends on the first pulldown list -sourcetypes to search from-. Uses a static lookup csv table, which is searched in postProcess param using the previous pulldown result as argument -$fuente$-. The postprocess lookup returns labelnames and values for the search params valid for the selected sourcetype. In free text search, this part of the resultant seach query will be empty -->
<module name="Pulldown">
 <param name="float">left</param>
 <param name="name">valor</param>
 <param name="label">Parametro de busqueda:</param>

 <param name="postProcess">
    <![CDATA[
     | inputlookup file.csv | where fuente="$fuente$" | fields nombre valor 
    ]]>
 </param>
 <param name="staticOptions"/>

 <!-- Equal symbol to avoid issues in free text search (no sourcetype is specified in the first pulldown list). -->
 <param name="template">$value$ =</param>
 <param name="valueField">valor</param>
 <param name="labelField">nombre</param

When I load this view, the second pulldown population through postProcess takes a lot of time (almost 7-8 seconds). I'm on the last Sideview Utils version on Splunk 5.0. Any idea of what could be the issue?

Thanks and regards!

SplunkTrust
SplunkTrust

Yes I know what the problem is. A postprocess search always exists in relation to some base search and is meaningless without some base search.

Here you are using a postprocess search to fill the second Pulldown but there is no base search. Unfortunately there is always an implied base search of "*" over all time. So you see the problem. 😃 The dynamic Pulldown tells the ui framework "I require search results", and so the ui framework obligingly dispatches a search for it, with a dispatch point at the level of that Pulldown module. However the search dispatched is "*", over all time. (!!!) Which is bad because this search can take an extremely long time depending on how much data you have indexed.

It is of course easy to forget this when the postprocess search is itself a generating command like inputlookup.

The answer is simply to move your postprocess param into a search module:

<module name="Pulldown">
  <param name="name">fuente</param>
  ...

  <module name="Search">
    <param name="search">| inputlookup file.csv | where fuente="$fuente$" | fields nombre valor</param>

    <module name="Pulldown">
      <param name="name">valor</param>
      ...

and to not use the Pulldown module's postprocess param at all here.

0 Karma

Path Finder

Allright, you nailed it! Your explanation absolutely makes sense, I will stream the pulldown after a search.

Thank you!

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!