Alerting

Prevent repeat alerts

Builder

Hi. We have script that Splunk runs every 15 minutes. The script checks to see if a partition is using the primary or back-up ethernet adapter. If it is using the back-up adapter, it notifies us via e-mail.

This alert works great except for one thing: it will repeat the alert every time the script runs. That means we get alerted every 15 minutes, which creates a cluttered mailbox in the morning.

I realize that this may be expected behavior. I'm wondering if anyone has a procedure or trick to prevent a repeat alert from going out.

I'm running Splunk 4.1.5 on the indexer and 4.1.4 on the forwarders.

Thanks!

Tags (2)
1 Solution

Motivator

Take a look at the AlertThrottle app on Splunkbase, with it you will be able to throttle repeat alerts.

View solution in original post

Explorer

How did you set it up so that it didn't notify you anymore? I've only been able to throttle it, not stop the notification all together.

0 Karma

Motivator

Take a look at the AlertThrottle app on Splunkbase, with it you will be able to throttle repeat alerts.

View solution in original post

Builder

🙂
Actually, I went ahead and tried it anyways. I must admit it worked like a charm. Thank you for the tip!

0 Karma

Motivator

@Branden, the AlertThrottle app was written and is maintained by a Splunk employee. Not sure if that alleviates any reservations, but maybe it will 🙂

0 Karma

Builder

Thank you for the response. I'm a bit wary about relying on a 3rd party app for our production system (it is poorly documented, has no ratings, and I'm not sure how reliable it is).
Nevertheless, I am willing to give it a try. Unfortunately, I am not sure I can create the custom condition I'm looking for in the custom condition field. I'm sure it can be done, I'll just have to ask it in a separate question.
Thank you again.

0 Karma