90th percentile search results

[amitsehgal]

I need to get average 90th percentile of my results from response time.

let say if there are 200 data points; I need to ignore top 20 and take an average of 90%.

Something very similar in excel : http://office.microsoft.com/en-us/excel-help/percentile-HP005209211.aspx

Kind of reverse top .I tried perc90(response_time) on stats but this ould calulate X=95% of Y and not ignore the entire range.

Thanks

[amitsehgal]

I've used as suggested but its too slow....it takes more than 15 mins to get the one hour of logs results.......... eventstats perc90(response_time) as response_time_90p | where response_time < response_time_90p | stats avg(response_time)

Is there any other way around to find the percentile ?

[amitsehgal]

Not sure how to bump this.....is there any suggestion to make this search 90 percentile faster...its too slow ....

[gkanapathy]

Unclear to me, do you want the value of the 90th percentile of your set, or do you want to discard the highest 10% and take the mean of the remainder? (And I'm not sure of the statistical usefulness of the latter.)

I suppose something else you can do is use the outlier search command to discard extreme values, then take a mean.

[amitsehgal]

I want to discard the top 10% of results and take a mean of 90%. Can outlier discard on fixed percentage ?

[gkanapathy]

Unclear to me, do you want the value of the 90th percentile of your set, or do you want to discard the highest 10% and take the mean of the remainder? (And I'm not sure of the statistical usefulness of the latter.)

[Stephen_Sorkin]

CORRECTED: given that you want the average excluding values greater than the 90th percentile.

You can use the "perc" aggregator of the stats command. Assuming that you have a field called "response_time", your search would be:

... | eventstats perc90(response_time) as response_time_90p | where response_time < response_time_90p | stats avg(response_time)

[Stephen_Sorkin]

This will give you the 90th percentile response time. That means it will take all response times, sort, and take the value 90% of the way from min to max. In this example, the 90th percentile is 9. If you want to find the average excluding the 90th percentile, then you need to search like: ... | eventstats perc90(response_time) as response_time_90perc | where response_time < response_time_90perc | stats avg(response_time)

[amitsehgal]

i meant i;m expecting [90% of 29]/8

[amitsehgal]

Thanks for the prompt response.I have tried perc 90; would it aggregate on overall avg response time or the series. e.g. for s series of response [1,2,3,4,5,6,7,8,9,10] would it give me [90% of 55]/10 OR ignore 9 and 10 and return [90% of 29]/8. I'm expecting later.