Splunk Search

90th percentile search results

Path Finder

I need to get average 90th percentile of my results from response time.

let say if there are 200 data points; I need to ignore top 20 and take an average of 90%.

Something very similar in excel : http://office.microsoft.com/en-us/excel-help/percentile-HP005209211.aspx

Kind of reverse top .I tried perc90(response_time) on stats but this ould calulate X=95% of Y and not ignore the entire range.

Thanks

Tags (2)

Path Finder

I've used as suggested but its too slow....it takes more than 15 mins to get the one hour of logs results.......... eventstats perc90(response_time) as response_time_90p | where response_time < response_time_90p | stats avg(response_time)

Is there any other way around to find the percentile ?

0 Karma

Path Finder

Not sure how to bump this.....is there any suggestion to make this search 90 percentile faster...its too slow ....

0 Karma

Splunk Employee
Splunk Employee

Unclear to me, do you want the value of the 90th percentile of your set, or do you want to discard the highest 10% and take the mean of the remainder? (And I'm not sure of the statistical usefulness of the latter.)

I suppose something else you can do is use the outlier search command to discard extreme values, then take a mean.

0 Karma

Path Finder

I want to discard the top 10% of results and take a mean of 90%. Can outlier discard on fixed percentage ?

0 Karma

Splunk Employee
Splunk Employee

Unclear to me, do you want the value of the 90th percentile of your set, or do you want to discard the highest 10% and take the mean of the remainder? (And I'm not sure of the statistical usefulness of the latter.)

0 Karma

Splunk Employee
Splunk Employee

CORRECTED: given that you want the average excluding values greater than the 90th percentile.

You can use the "perc" aggregator of the stats command. Assuming that you have a field called "response_time", your search would be:

... | eventstats perc90(response_time) as response_time_90p | where response_time < response_time_90p | stats avg(response_time)

Splunk Employee
Splunk Employee

This will give you the 90th percentile response time. That means it will take all response times, sort, and take the value 90% of the way from min to max. In this example, the 90th percentile is 9. If you want to find the average excluding the 90th percentile, then you need to search like: ... | eventstats perc90(responsetime) as responsetime90perc | where responsetime < responsetime90perc | stats avg(response_time)

0 Karma

Path Finder

i meant i;m expecting [90% of 29]/8

0 Karma

Path Finder

Thanks for the prompt response.I have tried perc 90; would it aggregate on overall avg response time or the series. e.g. for s series of response [1,2,3,4,5,6,7,8,9,10] would it give me [90% of 55]/10 OR ignore 9 and 10 and return [90% of 29]/8. I'm expecting later.

0 Karma