Splunk Search

How to alert on a value crossing over a moving average of that value?

responsys_cm
Builder

Is there any way to do this in a single search? I know it can be done by having one search compute the moving average of the field and then write that to a lookup table. A subsequent search would then compute the value for the time period and see if it is above the value in the lookup table.

Is there any way to do it with a single search that doesn't require a lookup table?

Thx.

Craig

Tags (2)
0 Karma

mhamano
Explorer

I'm trying to run a similar search looking at decline rates for a payment gateway. I want to look at the current decline rate average trend line and also the running average trendline.

tried doing something similar but ended up getting Mismatched ']' when I ran my query.

gateway=firstdata errorType=declined event=transaction Completed earliest=-1h@h latest=@h | bucket _time span=1h | stats count as hourlyCount by _time | appendcols { search searchforerrors earliest=-7d@d latest=-1h@h | bucket _time span=1h | stats count by _time | stats avg(count) as average ] |where hourlyCount > average

Any idea why?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

actually trendline or the streamstats commands will let you do this fairly easily, but really it's hard to say what works for you unless we know how you are averaging your data.

lguinn2
Legend

Let me make this more specific, so that i can show an example. Every hour, I want to compare the number of errors from the past hour with the average number of errors for the past week. If the hourly count > the average count, then trigger an alert.

This search will do

searchforerrors earliest=-1h@h latest=@h
| bucket _time span=1h
| stats count as hourlyCount by _time
| appendcols { search  searchforerrors earliest=-7d@d latest=-1h@h
      | bucket _time span=1h
      | stats count by _time
      | stats avg(count) as average ]
| where hourlyCount > average

Schedule this search to run once each hour and to trigger if the number of results is greater than zero.

mhamano
Explorer

I'm trying to run a similar search looking at decline rates for a payment gateway. I want to look at the current decline rate average trend line and also the running average trendline.

tried doing something similar but ended up getting Mismatched ']' when I ran my query.

gateway=firstdata errorType=declined event=transaction Completed earliest=-1h@h latest=@h | bucket _time span=1h | stats count as hourlyCount by _time | appendcols { search searchforerrors earliest=-7d@d latest=-1h@h | bucket _time span=1h | stats count by _time | stats avg(count) as average ] |where hourlyCount > average

Any idea why?

0 Karma
Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...