Splunk Search

How to search the 90th percentile value in a series of values and the count of values that are greater than the 90th percentile?

edookati
Path Finder

I need the 90th percentile value in a series of values and the count of values that are greater than the 90th percentile...
I am trying the below query with no luck. Please help me.

index=jms_logs sourcetype=perflogs domain_server_port="proda_olb_osb*" service_name="ABC*" | eventstats perc95(response_time) as response_time_95p | stats count by service_name | where response_time>response_time_98p

Tags (3)
0 Karma
1 Solution

lguinn2
Legend

Try this

index=jms_logs sourcetype=perflogs domain_server_port="proda_olb_osb*" service_name="ABC*" 
| eventstats perc90(response_time) as response_time_90p 
| where response_time>response_time_90p
| stats count by service_name

First, you had a typo, but you also had the commands a bit out of order. This calculates the 90th percentile of response time across all services and events, then counts the events that exceed the 90th percentile by service. If you want the 90th percentile to also be based on the service, rather than overall, then try this

index=jms_logs sourcetype=perflogs domain_server_port="proda_olb_osb*" service_name="ABC*" 
| eventstats perc90(response_time) as response_time_90p  by service_name
| where response_time>response_time_90p
| stats count by service_name

View solution in original post

0 Karma

lguinn2
Legend

Try this

index=jms_logs sourcetype=perflogs domain_server_port="proda_olb_osb*" service_name="ABC*" 
| eventstats perc90(response_time) as response_time_90p 
| where response_time>response_time_90p
| stats count by service_name

First, you had a typo, but you also had the commands a bit out of order. This calculates the 90th percentile of response time across all services and events, then counts the events that exceed the 90th percentile by service. If you want the 90th percentile to also be based on the service, rather than overall, then try this

index=jms_logs sourcetype=perflogs domain_server_port="proda_olb_osb*" service_name="ABC*" 
| eventstats perc90(response_time) as response_time_90p  by service_name
| where response_time>response_time_90p
| stats count by service_name
0 Karma

edookati
Path Finder

Also, i would like to set limits on 90p value for individual services (there are about 60-70) and alert if the 90p value goes above the set limit or if the count above 90p is higher than set limit.

Can you please suggest how this can be achieved.
Thanks a lot.

0 Karma

edookati
Path Finder

it worked. Thanks a lot.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...