Splunk Search

90th percentile search results

amitsehgal
Path Finder

I need to get average 90th percentile of my results from response time.

let say if there are 200 data points; I need to ignore top 20 and take an average of 90%.

Something very similar in excel : http://office.microsoft.com/en-us/excel-help/percentile-HP005209211.aspx

Kind of reverse top .I tried perc90(response_time) on stats but this ould calulate X=95% of Y and not ignore the entire range.

Thanks

Tags (2)

amitsehgal
Path Finder

I've used as suggested but its too slow....it takes more than 15 mins to get the one hour of logs results.......... eventstats perc90(response_time) as response_time_90p | where response_time < response_time_90p | stats avg(response_time)

Is there any other way around to find the percentile ?

0 Karma

amitsehgal
Path Finder

Not sure how to bump this.....is there any suggestion to make this search 90 percentile faster...its too slow ....

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Unclear to me, do you want the value of the 90th percentile of your set, or do you want to discard the highest 10% and take the mean of the remainder? (And I'm not sure of the statistical usefulness of the latter.)

I suppose something else you can do is use the outlier search command to discard extreme values, then take a mean.

0 Karma

amitsehgal
Path Finder

I want to discard the top 10% of results and take a mean of 90%. Can outlier discard on fixed percentage ?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Unclear to me, do you want the value of the 90th percentile of your set, or do you want to discard the highest 10% and take the mean of the remainder? (And I'm not sure of the statistical usefulness of the latter.)

0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

CORRECTED: given that you want the average excluding values greater than the 90th percentile.

You can use the "perc" aggregator of the stats command. Assuming that you have a field called "response_time", your search would be:

... | eventstats perc90(response_time) as response_time_90p | where response_time < response_time_90p | stats avg(response_time)

Stephen_Sorkin
Splunk Employee
Splunk Employee

This will give you the 90th percentile response time. That means it will take all response times, sort, and take the value 90% of the way from min to max. In this example, the 90th percentile is 9. If you want to find the average excluding the 90th percentile, then you need to search like: ... | eventstats perc90(response_time) as response_time_90perc | where response_time < response_time_90perc | stats avg(response_time)

0 Karma

amitsehgal
Path Finder

i meant i;m expecting [90% of 29]/8

0 Karma

amitsehgal
Path Finder

Thanks for the prompt response.I have tried perc 90; would it aggregate on overall avg response time or the series. e.g. for s series of response [1,2,3,4,5,6,7,8,9,10] would it give me [90% of 55]/10 OR ignore 9 and 10 and return [90% of 29]/8. I'm expecting later.

0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...