Alerting

How to Trigger alert for first 3 times and then suppress the consecutive alerts

ksubramanian198
Engager

Hi,
I have scheduled a Splunk alert to be executed for every 1 minute, if it matches my search condition for last 10 minutes events, then this will trigger an alert once. Throttle time is set as 8 minutes.
I would like to trigger 3 consecutive alerts and then throttle for 8 minutes, currently it triggers alert once and throttling for 8 minutes. Please let me know if there is a way achieve this.

Tags (1)
0 Karma

woodcock
Esteemed Legend

You can dump your conditions into a lookup and then create an independent caretaker/housekeeping/alerting search to clean it. Like this:

Your Current Alerting Search
| table _time Other Stuff Here
| outputlookup append=true YourLookup
| where ThisFIeldWillNeverExists=="So this line drops all current events"
| inputlookup append=true YourLookup
| eventstats count
| where count<=YourThrottlingLimitHere

Then you schedule your other search to do something like this:

|inputlookup YourLookup
| where _time <= relative_time(now, "-10m")
| outputlookup YourLookup

Tune to your liking but you get the idea.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...