Alerting

How to throttle alerts for 15 min delay?

sagar_shubham
Explorer

I have used this query for the alert creation.
index = xyz sourcetype=abc |table _time response_time|search response_time>50

I have used corn schedule for 5 min. But this creates lot of noise. So I want to use throttle for this alert for 15 min. Means after the first alerts triggered, it will take a 15 mins dealy.

I have used below configuration for each result triggered.

Throttle : "Checked"
Suppress results containing field value: "response_time"
Suppress triggering for : 15 mins

But this is not working. Please help.

andreacorvini
Path Finder

Throttle works for the same field value.
If the response time change, then you'll receive a new alert.
Leave the field blank.

0 Karma

sagar_shubham
Explorer

Sir, under Throttle what is the meaning of Suppress results containing field value?

0 Karma

sagar_shubham
Explorer

and how we can use that?

0 Karma

andreacorvini
Path Finder

As in your example you can use "Server" in "Suppress results containing field value" if you want to stop alerts for the same server (i.e. all alerts for Server srv05).
If you wan to stop alerts for all servers, leave blank "Suppress results containing field value".

Time Server Response_time
07:40 srv01 28
07:58 srv05 58
08:50 srv04 13
10:13 srv08 43
11:54 srv03 33

0 Karma

sagar_shubham
Explorer

Thanks Sir.

0 Karma

adonio
Ultra Champion

what is the purpose of the alert?
what is the trigger?
try to filter early, something like this
index = xyz sourcetype=abc response_time>50
and then do your function or rule, maybe like that:
bin _time span=1m | stats count as count_of_response_time_greater_than_50 by _time
than alert on condition, for example: count_of_response_time_greater_than_50 > X
if you need throttling after that, use the alert setup wizard to set it up

0 Karma

sagar_shubham
Explorer

Sir, here we are creating an alert, whose response time is greater than 50sec. The response time is very dynamic. So for a particular time there are lots of servers and we are calculating the response time for each of the server.

Time Server Response_time
07:40 srv01 28
07:58 srv05 58
08:50 srv04 13
10:13 srv08 43
11:54 srv03 33

The alert will be triggered in every 5 mins. But while using this 5 min there are lots of noise. So here i need
to configure after the first alerts triggered, it will take a 15 mins delay.

0 Karma

andreacorvini
Path Finder

In this case you can use "Server" field to throttle if you want to stop alerts for that.
Otherwise leave it blank to stop all.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...