Splunk Search

How do I set an alert when 95th percentile is over the threshold?

mlui_2
Explorer

Hi all,

Here is my base search

index=java  location=APICall api_method=POST Duration |stats   median(Duration) as median, perc70(Duration) as p70, perc90(Duration) as p90, perc95(Duration) as p95, perc99(Duration) as p99 

How do I set an alert when 95th percentile is over let's say 30000ms ?

Thanks in advance

1 Solution

pjdwyer
Explorer

If I understand your data correctly you would want to add a where statement before the stats statement.

| where perc95(duration) > 30000

and if you want to add more clauses to the statement you can do it this way:

| where (perc70(duration) > 30000) OR (perc90(duration) > 30000) OR (perc95(duration) > 30000) OR (perc99(duration) > 30000)

This statement will remove any information that does not meet the said criteria.

View solution in original post

mlui_2
Explorer

index=java location=APICall api_method=POST Duration |timechart span=5m perc95(Duration) as P95| where P95> 30000ms

0 Karma

mlui_2
Explorer

got it

timechart span=5m perc95(Duration) as P95 | where P95>30000ms

0 Karma

pjdwyer
Explorer

If I understand your data correctly you would want to add a where statement before the stats statement.

| where perc95(duration) > 30000

and if you want to add more clauses to the statement you can do it this way:

| where (perc70(duration) > 30000) OR (perc90(duration) > 30000) OR (perc95(duration) > 30000) OR (perc99(duration) > 30000)

This statement will remove any information that does not meet the said criteria.

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...