Splunk Search

How to generate a search to find the number of days that exceeds mean by certain ranges?

jrnastase
Explorer

Hello all!

I'm trying to find the number of days that the daily count of my event exceeds the daily mean + standard deviation for a 3-week period. I also need to return the number of days that exceeds the mean + 2 stdevs and mean + 3 stdevs, and keep it all together.

Is there an easy way to do this?

0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust
index="index" field="field" sourcetype="sourcetype"
| bucket _time span=1d 
| stats count as EventCount by _time

| rename COMMENT as "This section adds records for the days which had zero counts.  Remove if unwanted."
| appendpipe [| stats min(_time) as mintime max(_time) as maxtime | eval _time=mvrange(mintime,maxtime,86400) | eval EventCount=0]
| stats max(EventCount) as EventCount by _time

| rename COMMENT as "This section calculates which days were beyond n stdevs, and sets a flag to count them up."  
| eventstats avg(EventCount) as avgEventCount stdev(EventCount) as stdevEventCount 
| eval logs1=if(EventCount> avgEventCount+1*stdevEventCount,1,0)
| eval logs2=if(EventCount> avgEventCount+2*stdevEventCount,1,0)
| eval logs3=if(EventCount> avgEventCount+3*stdevEventCount,1,0)

| rename COMMENT as "This section adds records to show the specific dates beyond n stdevs, for test purposes.  Remove when working if unwanted."  
| eval day = strftime(_time,"%Y-%m-%d")
| eval days1=if(EventCount> avgEventCount+1*stdevEventCount,day,0)
| eval days2=if(EventCount> avgEventCount+2*stdevEventCount,day,0)
| eval days3=if(EventCount> avgEventCount+3*stdevEventCount,day,0)

| rename COMMENT as "This section calculates and reports your answers."  
| stats sum(log*) as log*, values(day*) as day* 

View solution in original post

woodcock
Esteemed Legend

Check out this Q&A for a very in-depth conversation on this topic (don't forget to up-vote):
https://answers.splunk.com/answers/511894/how-to-use-the-timewrap-command-and-set-an-alert-f.html#an...

DalJeanis
SplunkTrust
SplunkTrust
index="index" field="field" sourcetype="sourcetype"
| bucket _time span=1d 
| stats count as EventCount by _time

| rename COMMENT as "This section adds records for the days which had zero counts.  Remove if unwanted."
| appendpipe [| stats min(_time) as mintime max(_time) as maxtime | eval _time=mvrange(mintime,maxtime,86400) | eval EventCount=0]
| stats max(EventCount) as EventCount by _time

| rename COMMENT as "This section calculates which days were beyond n stdevs, and sets a flag to count them up."  
| eventstats avg(EventCount) as avgEventCount stdev(EventCount) as stdevEventCount 
| eval logs1=if(EventCount> avgEventCount+1*stdevEventCount,1,0)
| eval logs2=if(EventCount> avgEventCount+2*stdevEventCount,1,0)
| eval logs3=if(EventCount> avgEventCount+3*stdevEventCount,1,0)

| rename COMMENT as "This section adds records to show the specific dates beyond n stdevs, for test purposes.  Remove when working if unwanted."  
| eval day = strftime(_time,"%Y-%m-%d")
| eval days1=if(EventCount> avgEventCount+1*stdevEventCount,day,0)
| eval days2=if(EventCount> avgEventCount+2*stdevEventCount,day,0)
| eval days3=if(EventCount> avgEventCount+3*stdevEventCount,day,0)

| rename COMMENT as "This section calculates and reports your answers."  
| stats sum(log*) as log*, values(day*) as day* 

jrnastase
Explorer

If it helps, this is what I tried before, but it only works if values exist for logs++

index="index" field = "field" sourcetype="sourcetype"
| bucket _time span=1d
| stats count by _time
| eventstats avg(count) as average stdev(count) as standard_deviation

| where count>average+standard_deviation
| eventstats count as logs
| where count>average+(2*standard_deviation)
| eventstats count as logs+
| where count>average+(3*standard_deviation)
| eventstats count as logs++

0 Karma
Get Updates on the Splunk Community!

Splunk Lantern | Spotlight on Security: Adoption Motions, War Stories, and More

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Cloud | Empowering Splunk Administrators with Admin Config Service (ACS)

Greetings, Splunk Cloud Admins and Splunk enthusiasts! The Admin Configuration Service (ACS) team is excited ...

Tech Talk | One Log to Rule Them All

One log to rule them all: how you can centralize your troubleshooting with Splunk logs We know how important ...