topic Re: post-process not delivering all events to chart? in Splunk Search

post-process not delivering all events to chart?

sf_user_199 — Thu, 29 Sep 2011 17:48:48 GMT

I have the following xml

<module name="HiddenSearch" layoutPanel="panel_row2_col1" group="XXX" autoRun="True"> <param name="search">apache log search method=get OR method=post | eval responseStatus=case(match(status,"2\d\d"),"OK",match(status,"3\d\d"),"OK",match(status,"4\d\d"),"ERROR",match(status,"5\d\d"),"ERROR") </param> <param name="earliest">-30m@m</param> <module name="HiddenPostProcess" layoutPanel="panel_row2_col1_grp1"> <param name="search">timechart count by status</param> <module name="HiddenChartFormatter"> <param name="charting.chart">column</param> <param name="charting.chart.stackMode">stacked</param> <param name="charting.legend.placement">bottom</param> <module name="JobProgressIndicator"/> <module name="FlashChart"> <param name="width">100%</param> <param name="enableResize">True</param> <module name="ConvertToDrilldownSearch"> <module name="ViewRedirector"> <param name="viewTarget">flashtimeline</param> </module> </module> <module name="ViewRedirectorLink"> <param name="viewTarget">flashtimeline</param> </module> </module> </module> </module> <module name="HiddenPostProcess" layoutPanel="panel_row2_col1_grp2"> <param name="search">top responseStatus | where match(responseStatus,"ERROR") | gauge percent 0 5 10 100</param> <module name="HiddenChartFormatter"> <param name="charting.chart">fillerGauge</param> <param name="charting.chart.style">shiny</param> <param name="charting.chart.orientation">x</param> <param name="charting.chart.usePercentageRange">true</param> <param name="charting.chart.usePercentageValue">true</param> <module name="JobProgressIndicator"/> <module name="FlashChart"> <param name="width">100%</param> <param name="enableResize">False</param> <module name="ConvertToDrilldownSearch"> <module name="ViewRedirector"> <param name="viewTarget">flashtimeline</param> </module> </module> </module> <module name="ViewRedirectorLink"> <param name="viewTarget">flashtimeline</param> </module> </module> </module> </module>

What happens is that the gauge from TOP is correctly displayed, but the timechart only gets 5 minutes of data. It should be displaying all 30 minutes of data. Any suggestions?

ETA: If you change the timechart to chart count by _time then the chart will only display a 5 minute graph, vs a 30 minute graph with 5 minutes of data with timechart.

Re: post-process not delivering all events to chart?

bwooden — Thu, 29 Sep 2011 19:25:46 GMT

I think you'll want to switch your top & your where so that you are getting a top of ERRORs instead of subset of ERRORs from whatever topped.

from

<param name="search">top responseStatus | where match(responseStatus,"ERROR") | gauge percent 0 5 10 100</param>

<param name="search">where match(responseStatus,"ERROR") | top responseStatus | gauge percent 0 5 10 100</param>

Re: post-process not delivering all events to chart?

sf_user_199 — Thu, 29 Sep 2011 19:44:44 GMT

Unless you think that this is affecting the timechart postprocess, the change you are suggesting doesn't help. I only want the subset of errors.

Re: post-process not delivering all events to chart?

bwooden — Thu, 29 Sep 2011 20:03:27 GMT

Yeah, I misread the problem/question

Re: post-process not delivering all events to chart?

sideview — Fri, 30 Sep 2011 03:26:19 GMT

You should check out the docs around postProcess. In particular if the base search does not contain any transforming search commands splunk will not preserve full information about the events past the 50,000'th event.

Check out the UI Examples app on Splunkbase, and read the page called "Using postProcess on dashboards".

or check out the docs here. http://docs.splunk.com/Documentation/Splunk/latest/Developer/PostProcess

The answer in your case is as follows:

instead of having this search

apache log search method=get OR method=post | eval responseStatus=case(match(status,"2\d\d"),"OK",match(status,"3\d\d"),"OK",match(status,"4\d\d"),"ERROR",match(status,"5\d\d"),"ERROR")

You want to have this base search:

apache log search method=get OR method=post | eval responseStatus=case(match(status,"2\d\d"),"OK",match(status,"3\d\d"),"OK",match(status,"4\d\d"),"ERROR",match(status,"5\d\d"),"ERROR") | bin _time span="1min" | stats count by _time responseStatus

Splunk basically will not keep arbitrarily large numbers of events around. On the other hand if the search has transforming commands, it will assemble a complete result set and not cut any corners. Adding these bin and stats commands means that you'll have a much more compact and efficient data set to work with, with no missing information.

Re: post-process not delivering all events to chart?

sf_user_199 — Mon, 28 Sep 2020 09:59:59 GMT

This worked! The only downside to this is I had to do some trickery to get results similar to the top command.