Sideutils Postprocess module + splunk nullmodule = postprocess search not performed.

[Lucas_K]

Sideutils version : 2.4.7 (can't upgrade to latest version until next outage window due to this being on a deployment server which will restart a large number of other machines).
Splunk version : 5.0.4

I have a sideutils postprocess search that prior to 5.0.4 worked without a problem. Just wondering if this is a known bug with either splunk or sideutils.

Details, I have a dashboard that has a base search that produces a table. I then use a postprocess module to filter out only red exception events and save them to a lookup file. This postprocess module then feeds to a child nullmodule as it has no displayable output. This filtered csv lookup is used in another dashboard to roll up several other dashboard searches into a single exception report without having to re-run expensive searches. The main reason for using a nullmodule was that using a blank table resulted in a
"waiting for results" message within emailed PDF reports. By using nullmodule as the child of the postprocess I was able to hide this in the email.

Since upgrading to 5.0.4 the postprocess doesn't seem to run at all when a nullmodule is used. If I change it back to a table (which will be empty) then its fine and the lookup is created.

Sample xml. (i'm not quite sure how to paste xml directly so the web form takes it exactly as is 😕 )

    <module name="Search" layoutPanel="panel_row1_col1" group="Indexing Report - Last 30 Minutes" autoRun="True">
   <param name="search">
  [CDATA[ 
       index="_internal" sourcetype="splunkd" group="per_index_thruput" NOT (series="_*" OR series="/*" OR series="summary*") | rename series as index | stats count by _time, index, kbps | append [ search index="_internal" sourcetype="splunkd" LicenseUsage NOT (s="" OR s="_*" OR s="/*" OR s="summary*" OR s="tcp*" OR s="udp*") earliest=-30m@m | eval kbps=b/1024 | rename s as index | stats count by _time, index, kbps] | append [inputlookup indexes.csv | fields index] | stats avg(kbps) by index | fillnull value=0 | lookup health_indexes.csv index OUTPUT input, forwarder, comment, alert  | fillnull value=0 | search input="Realtime" AND alert="Yes" | rename index AS Index, forwarder AS "Forwarder", comment AS "Data Source", avg(kbps) as kbps | fields - alert, input | sort kbps | eval myClassFieldD=case(kbps>0.001,"green",kbps==0,"red")
  ]]
  < /param>
  <param name="earliest">-30m@m< /param>
  <param name="latest">now< /param>
  <module name="Table">
  <param name="count">10</param>
  <param name="rowClass">$row.fields.myClassFieldD$< /param> 
  <param name="hiddenFields">myClassFieldD< /param>
  </module>
  <module name="PostProcess" layoutPanel="panel_row1_col1" group="Indexing Report - Last 30 Minutes">
<param name="search">search myClassFieldD=red | eval now=now() | eval description="Missing Index events in last 30 mins" | eval Dashboard="Index_Status" | eval DashboardDescription="REPORTS : Index Status" | eval section="indexing" | append [|stats count |eval count="null_value"| rename count as "info_search_marker" ] | outputlookup index_status_indexing_report</param>
<module name="NullModule" />                                
  < /module> 

Thanks 🙂

[sideview · Accepted solution]

I'm not sure how this was ever working, but I believe you!

The fix is to take postProcess out of this equation entirely, replace it with a second search, and then use a CustomBehavior module to get the framework to kick off your invisible search.

Here's modified XML - note that instead of PostProcess + NullModule there is a Search + CustomBehavior. Also note that the Search actually references the upstream search with $search$, as this is key.

    <module name="Search" layoutPanel="panel_row1_col1" group="Indexing Report - Last 30 Minutes" autoRun="True">
  <param name="search"><![CDATA[
     index="_internal" sourcetype="splunkd" group="per_index_thruput" NOT (series="_*" OR series="/*" OR series="summary*") | rename series as index | stats count by _time, index, kbps | append [ search index="_internal" sourcetype="splunkd" LicenseUsage NOT (s="" OR s="_*" OR s="/*" OR s="summary*" OR s="tcp*" OR s="udp*") earliest=-30m@m | eval kbps=b/1024 | rename s as index | stats count by _time, index, kbps] | append [inputlookup indexes.csv | fields index] | stats avg(kbps) by index | fillnull value=0 | lookup health_indexes.csv index OUTPUT input, forwarder, comment, alert  | fillnull value=0 | search input="Realtime" AND alert="Yes" | rename index AS Index, forwarder AS "Forwarder", comment AS "Data Source", avg(kbps) as kbps | fields - alert, input | sort kbps | eval myClassFieldD=case(kbps>0.001,"green",kbps==0,"red")
  ]]>< /param>
  <param name="earliest">-30m@m< /param>
  <param name="latest">now< /param>

  <module name="Table">
    <param name="count">10</param>
    <param name="rowClass">$row.fields.myClassFieldD$< /param> 
    <param name="hiddenFields">myClassFieldD< /param>
  </module>

  <module name="Search" layoutPanel="panel_row1_col1" group="Indexing Report - Last 30 Minutes">
    <param name="search">$search$ | search myClassFieldD=red | eval now=now() | eval description="Missing Index events in last 30 mins" | eval Dashboard="Index_Status" | eval DashboardDescription="REPORTS : Index Status" | eval section="indexing" | append [|stats count |eval count="null_value"| rename count as "info_search_marker" ] | outputlookup index_status_indexing_report</param>

    <module name="CustomBehavior">
      <param name="requiresDispatch">True</param>
    </module>
</module>

[Lucas_K]

Just tried it and that last one works. I think it hides the "waiting for results" in the pdf too.

[sideview]

Hm. you're right. I don't know why I didn't think of that. It's definitely an issue.

OK. If you put the PostProcess back, you can replace the CustomBehavior with....

<module name="ResultsValueSetter">
   <param name="fields">foo</param>
   <module name="CustomBehavior"></module>
</module>

which is even weirder. What we have to do is ensure something makes a request. Your NullModule wouldn't have done it. Here ResultsValueSetter I think will make one and only one request, it'll pick up that postProcess and it'll work.

[Lucas_K]

haha.

Thanks again. Will have to change it and try it out 🙂

As it is a new search would this mean a full new search is created (as opposed how post process just reuses results from a previously run search).