topic Re: Alert Trigger Condition - Alert only if second string is not present in Alerting

Alert Trigger Condition - Alert only if second string is not present

pierrem — Mon, 09 May 2022 16:54:03 GMT

Hi All,

I'm currently trying to configure a alert to trigger when 2 events are NOT present in last 15min.
In short if we have only Event1 but not Event2 then a alert should be triggered, if both events are present in last 15min then no alerts should be triggered.

Use case, the alert is being configured to alert us when a VPN tunnel interface goes down and stays down for more than 15min, generally these VPN connections to terminate briefly but comes back up after a few seconds, hence we would like only alert if Event1 (down) took place in last 15min without Event2 (up) taking place.

Event1 - Search query

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND "Lost Service"

Event2 - Search query

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND (inbound "LAN-to-LAN" "created")

Search Query to show both events

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND ("Lost Service" OR (inbound "LAN-to-LAN" "created"))

Any assistance will be greatly appreciated 🙂

Re: Alert Trigger Condition - Alert only if second string is not present

ITWhisperer — Mon, 09 May 2022 08:52:06 GMT

You could try something like this

index=firewall 10.10.10.10 Firewall_Name_XYZ=TEST123 AND ("Lost Service" OR (inbound "LAN-to-LAN" "created")) | rex "(?<event>Lost Service)" | fillnull value="inbound created" event | stats latest(event) as lastevent latest(_time) as lasttime | where lastevent = "Lost Service" AND lasttime < now()-15*60

Re: Alert Trigger Condition - Alert only if second string is not present

pierrem — Wed, 11 May 2022 10:34:24 GMT

Thanks ITWhisperer

It works like a charm, I just removed the lasttime statement as the alert is configured to run in a cron schedule searching last 15min 🙂

Thanks for the quick assistance