https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130767#M35632 <P>So I am trying to filter out outliers using the 3 sigma rule across some transactions. My search is as follows:</P> <PRE><CODE>blah... blah... blah... | eval durationMins = (duration / 60) | appendpipe [stats median(durationMins) as Med avg(durationMins) as Avrg stdev(durationMins) as Stddev max(durationMins) as Maxm] | eval threeSigmaLimit = (Avrg + (Stddev * 3)) | where durationMins &lt; threeSigmaLimit </CODE></PRE> <P>If i remove the appendpipe command and the eval and where stuff at the end, i get a table with the correct values, but I want to store the average and standard deviation, perform the eval statement, and filter out events where the duration of the transaction is greater than threeSigmaLimit <EM>on each event</EM>...</P> <P>The above command doesn't work.... I can't even see the Med, Avrg, Stddev, and Maxm fields in the Events tab in verbose mode....</P> <P>Any ideas?</P> Fri, 21 Nov 2014 22:58:03 GMT nterry 2014-11-21T22:58:03Z https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130767#M35632 <P>So I am trying to filter out outliers using the 3 sigma rule across some transactions. My search is as follows:</P> <PRE><CODE>blah... blah... blah... | eval durationMins = (duration / 60) | appendpipe [stats median(durationMins) as Med avg(durationMins) as Avrg stdev(durationMins) as Stddev max(durationMins) as Maxm] | eval threeSigmaLimit = (Avrg + (Stddev * 3)) | where durationMins &lt; threeSigmaLimit </CODE></PRE> <P>If i remove the appendpipe command and the eval and where stuff at the end, i get a table with the correct values, but I want to store the average and standard deviation, perform the eval statement, and filter out events where the duration of the transaction is greater than threeSigmaLimit <EM>on each event</EM>...</P> <P>The above command doesn't work.... I can't even see the Med, Avrg, Stddev, and Maxm fields in the Events tab in verbose mode....</P> <P>Any ideas?</P> Fri, 21 Nov 2014 22:58:03 GMT https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130767#M35632 nterry 2014-11-21T22:58:03Z https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130768#M35633 <P>Change <CODE>stats</CODE> to <CODE>eventstats</CODE>. Furthermore, you don't need the appendpipe.</P> <PRE><CODE>| eval durationMins = (duration/60) | eventstats median(durationMins) as Med, avg(durationMins) as Avrg, stdev(durationMins) as StDev, max(durationMins) as Maxm | eval threeSigmaLimit = (Avrg + (StDev * 3)) | where durationMins &lt; threeSigmaLimit </CODE></PRE> <P>Eventstats command documentation <A href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Eventstats">here</A>.</P> Sat, 22 Nov 2014 00:23:31 GMT https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130768#M35633 aljohnson_splun 2014-11-22T00:23:31Z https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130769#M35634 <P>THANK YOU SO MUCH!!!!! I was banging my head against the wall for hours trying to figure this one out....</P> Mon, 24 Nov 2014 20:02:09 GMT https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130769#M35634 nterry 2014-11-24T20:02:09Z https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130770#M35635 <P>Quick question, do the fields Med, Avrg, etc contain their corresponding stats values after performing the where clause, or before. </P> <P>If before, would I pipe it to this?</P> <PRE><CODE>... | where durationMins &lt; threeSigmaLimit | stats median(durationMins) as "Median Duration" avg(durationMins) as "Average Duration" stdev(durationMins) as "Standard Deviation" max(durationMins) as "Max Duration" </CODE></PRE> Mon, 24 Nov 2014 21:57:01 GMT https://community.splunk.com/t5/Splunk-Search/Filter-out-outliers-in-stats-command/m-p/130770#M35635 nterry 2014-11-24T21:57:01Z