topic Re: How to stop a single account email alert to trigger multiple emails? in Alerting

How to stop a single account email alert to trigger multiple emails?

k45bryant — Sat, 01 Sep 2018 01:46:24 GMT

I configured an alert to send an email every time a user is added to the Domain Admins group. I have this alert triggering on eventcode 4728, 4755, etc. The problem is that adding a single account will trigger multiple emails. I want the first event to trigger an email, but all subsequent events not to trigger an email. How do I accomplish this?

Re: How to stop a single account email alert to trigger multiple emails?

burwell — Sun, 02 Sep 2018 01:02:12 GMT

What time period are you searching over? You are looking for any events that match those codes in what time period? Do the emails stop?

Re: How to stop a single account email alert to trigger multiple emails?

k45bryant — Sun, 02 Sep 2018 12:33:22 GMT

Hello Burwell,

It is a real-time configuration. As far as I know, the Splunk system monitors the logs and at the very time it receives a log with an EventCode of 4728 or 4755, it triggers an email. This process is instantaneous and not configured for a time period. Did I answer your question?

Re: How to stop a single account email alert to trigger multiple emails?

burwell — Mon, 03 Sep 2018 22:04:24 GMT

Hi. So I have always avoid real-time alerts because I understand scheduled alerts. It seems the advice is that if you need something to be run frequently, run every minute.

When I look at real-time alerts (I am on 6.6.8: https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/DefineRealTimeAlerts) there are two ways to go.

I created an alert with the per-result triggering and made the action be to add to list of triggered alerts (I didn't want to spam myself with email)

1) Create a real-time alert with per-result triggering
I created one of these and could see, as admin, that the alert had a cron schedule of * * * * * (i.e. every minute) but when I caused the search to match and I looked at the triggered alerts, it immediately fired exactly one

2) I followed the instructions and removed the per result action. Instead I chose Number of events greater than 0. There was then a window to schedule in. When I went to save I was told "windowed real-time per result alerts require field based alert throttling to be enabled" . I attempted to do that and was told that I had to fill in "supress results containing field value". I was lost.

In summary, I think it is way more obvious what is happening to not use a real-time alert. Schedule your alert to run perhaps every minute for maybe the last -2m to last -1m (giving yourself plenty of time to index the event.) The expected behavior is far more understandable.

Re: How to stop a single account email alert to trigger multiple emails?

burwell — Mon, 03 Sep 2018 22:04:47 GMT

I commented in my answer above. I think real-time alerts are too confusing.

Re: How to stop a single account email alert to trigger multiple emails?

k45bryant — Mon, 03 Sep 2018 23:06:55 GMT

Perhaps scheduling alerts would be the way to go. We don't have to know the second someone adds an account to domain admins, we just have to know in a reasonable time so we can verify if this action has been approved. Scheduling an alert once a minute should suffice. Can you explain to me how to accomplish scheduled alerts for what I am trying to do?

Thank you so much for your help with this.

Kenneth

Re: How to stop a single account email alert to trigger multiple emails?

burwell — Tue, 04 Sep 2018 06:18:36 GMT

Kenneth, you want to schedule an alert on a cron schedule:
https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/AlertSchedulingBestPractices

Earliest -2m
Latest -1m
Cron schedule would be * * * * *
That's every minute. You might only want to run the alert every 5 minutes. In which case you would

Earliest -6m
Latest -1m
Cron schedule: */5 * * * * which means run the alert every 5 minutes

Your search would be something that would include the codes you care about including the index
e.g. index= sourcetype= 4728 OR 4755

If this helps be sure to accept my answer. Thanks.