Prevent repeat alerts

[Branden]

Hi. We have script that Splunk runs every 15 minutes. The script checks to see if a partition is using the primary or back-up ethernet adapter. If it is using the back-up adapter, it notifies us via e-mail.

This alert works great except for one thing: it will repeat the alert every time the script runs. That means we get alerted every 15 minutes, which creates a cluttered mailbox in the morning.

I realize that this may be expected behavior. I'm wondering if anyone has a procedure or trick to prevent a repeat alert from going out.

I'm running Splunk 4.1.5 on the indexer and 4.1.4 on the forwarders.

Thanks!

[lmalhoit]

How did you set it up so that it didn't notify you anymore? I've only been able to throttle it, not stop the notification all together.

[ftk · Accepted solution]

Take a look at the AlertThrottle app on Splunkbase, with it you will be able to throttle repeat alerts.

[Branden]

🙂
Actually, I went ahead and tried it anyways. I must admit it worked like a charm. Thank you for the tip!

[ftk]

@Branden, the AlertThrottle app was written and is maintained by a Splunk employee. Not sure if that alleviates any reservations, but maybe it will 🙂

[Branden]

Thank you for the response. I'm a bit wary about relying on a 3rd party app for our production system (it is poorly documented, has no ratings, and I'm not sure how reliable it is).
Nevertheless, I am willing to give it a try. Unfortunately, I am not sure I can create the custom condition I'm looking for in the custom condition field. I'm sure it can be done, I'll just have to ask it in a separate question.
Thank you again.