Splunk Search

How to count the number of events that are greater than 2 Standard deviations from the mean

richnavis
Contributor

I have events with response times.. and I want to find out how many of these events have response times > 2 Standard Deviations.

Tags (1)
0 Karma
1 Solution

melonman
Motivator

Hi

A bit messy, but is this what you are looking for?

sourcetype=your_sourcetype [search sourcetype=your_sourcetype | stats stdev(reponse_time) as stdev | eval search="response_time>".(stdev*2) | fields + search] | stats count

View solution in original post

emiller42
Motivator

There is another approach that doesn't require a subsearch:

sourcetype=your_sourcetype  | streamstats avg(response_time) as average stdev(response_time) as standard_deviation  | where response_time>average+(2*standard_deviation)

streamstats lets you gather an aggregate but represent it as a field per event. So each event gets an 'average' field that is the rolling average to that point. (A moving average, basically) This is great for comparing event values to aggregates, which is what you want to do here.

Alternatively, you can use eventstats, which does the same thing in giving each event a field with an aggregate value, but the difference here is it gathers the aggregate for the entire result set.

Play around with them and see what they do.

melonman
Motivator

Hi

A bit messy, but is this what you are looking for?

sourcetype=your_sourcetype [search sourcetype=your_sourcetype | stats stdev(reponse_time) as stdev | eval search="response_time>".(stdev*2) | fields + search] | stats count

melonman
Motivator

That "." is the String addition.
so the search field in subsearch specified in [] will look something like:

response_time>1234.56

and passed the response_time>1234.56 to the main search to fileter the events.

0 Karma

richnavis
Contributor

Works perfectly.. Thanks! BTW.. what does the"." character do in the subsearch?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...