Alerting

Alert Trigger Condition - Alert only if second string is not present

pierrem
Engager

Hi All, 

I'm currently trying to configure a alert to trigger when 2 events are NOT present in last 15min. 
In short if we have only Event1 but not Event2 then a alert should be triggered, if both events are present in last 15min then no alerts should be triggered. 

Use case, the alert is being configured to alert us when a VPN tunnel interface goes down and stays down for more than 15min, generally these VPN connections to terminate briefly but comes back up after a few seconds, hence we would like only alert if Event1 (down) took place in last 15min without Event2 (up) taking place. 

Event1 - Search query

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND "Lost Service"



Event2 - Search query 

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND (inbound "LAN-to-LAN" "created")



Search Query to show both events 

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND ("Lost Service" OR (inbound "LAN-to-LAN" "created"))



Any assistance will be greatly appreciated 🙂 

Labels (1)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

You could try something like this

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND ("Lost Service" OR (inbound "LAN-to-LAN" "created"))
| rex "(?<event>Lost Service)"
| fillnull value="inbound created" event
| stats latest(event) as lastevent latest(_time) as lasttime
| where lastevent = "Lost Service" AND lasttime < now()-15*60

View solution in original post

pierrem
Engager

Thanks ITWhisperer 

It works like a charm, I just removed the lasttime statement as the alert is configured to run in a cron schedule searching last 15min 🙂 

Thanks for the quick assistance 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You could try something like this

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND ("Lost Service" OR (inbound "LAN-to-LAN" "created"))
| rex "(?<event>Lost Service)"
| fillnull value="inbound created" event
| stats latest(event) as lastevent latest(_time) as lasttime
| where lastevent = "Lost Service" AND lasttime < now()-15*60
Get Updates on the Splunk Community!

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...

Explore the Latest Educational Offerings from Splunk [January 2025 Updates]

At Splunk Education, we are committed to providing a robust learning experience for all users, regardless of ...

Developer Spotlight with Paul Stout

Welcome to our very first developer spotlight release series where we'll feature some awesome Splunk ...