Alerting

Alert based on evaluating two time ranges

nbharadwaj
Path Finder

I am trying to send alerts based on the following criteria:

Last 15-20 mins: Measure avg(field1) --> value1 Last 10-15 mins: Measure avg(field1) --> value2

Calculate the % change between value1 and value2:

diff = (value2-value1)*100/value1

If diff is more than X percent than I need to send an alert notification.

How can I do this in one search command?

1 Solution

sideview
SplunkTrust
SplunkTrust

In the TimeRangePicker, choose 'custom time', then go to 'advanced'.

Enter the following as your timerange.

earliest = -20min@min

latest = -10min@min

And here's a search that will work:

<your search terms> | eval period=if(_time<relative_time(now(), "-15min@min"),"older_period", "recent_period") | eval foo=1 | chart avg(field1) over foo by period | eval diff = (recent_period-older_period)*100/older_period | where diff>X

Then save the search, click the 'schedule this search' box, and tell splunkd that if the search returns 1 or more rows, that you want it to send you an email.

here's another different search that will work too, although you have to use the same timerange

<your search terms> | timechart avg(field1) as avg span=5min | transpose | search column=avg | rename "row 1" as "older_period" | rename "row 2" as "recent_period" | eval diff = (recent_period-older_period)*100/older_period | where diff>X

View solution in original post

sideview
SplunkTrust
SplunkTrust

In the TimeRangePicker, choose 'custom time', then go to 'advanced'.

Enter the following as your timerange.

earliest = -20min@min

latest = -10min@min

And here's a search that will work:

<your search terms> | eval period=if(_time<relative_time(now(), "-15min@min"),"older_period", "recent_period") | eval foo=1 | chart avg(field1) over foo by period | eval diff = (recent_period-older_period)*100/older_period | where diff>X

Then save the search, click the 'schedule this search' box, and tell splunkd that if the search returns 1 or more rows, that you want it to send you an email.

here's another different search that will work too, although you have to use the same timerange

<your search terms> | timechart avg(field1) as avg span=5min | transpose | search column=avg | rename "row 1" as "older_period" | rename "row 2" as "recent_period" | eval diff = (recent_period-older_period)*100/older_period | where diff>X

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk Add-on for Microsoft Windows 11.0.3: Parse Once, Search Faster

The updated Splunk Add-on for Microsoft Windows helps customers search Windows events faster by reducing ...

What's New in Splunk Enterprise Security 8.8 and Beyond

Threats move at machine speed. Keeping up should not depend on having the biggest budget, the largest team, or ...

A Four-Part Event Series: State of Observability - Where AI Meets Performance

As AI agents take on more complex tasks, teams need greater visibility into how they behave, what they cost, ...