Alerting

How to stop a single account email alert to trigger multiple emails?

New Member

I configured an alert to send an email every time a user is added to the Domain Admins group. I have this alert triggering on eventcode 4728, 4755, etc. The problem is that adding a single account will trigger multiple emails. I want the first event to trigger an email, but all subsequent events not to trigger an email. How do I accomplish this?

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Hi. So I have always avoid real-time alerts because I understand scheduled alerts. It seems the advice is that if you need something to be run frequently, run every minute.

When I look at real-time alerts (I am on 6.6.8: https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/DefineRealTimeAlerts) there are two ways to go.

I created an alert with the per-result triggering and made the action be to add to list of triggered alerts (I didn't want to spam myself with email)

1) Create a real-time alert with per-result triggering
I created one of these and could see, as admin, that the alert had a cron schedule of * * * * * (i.e. every minute) but when I caused the search to match and I looked at the triggered alerts, it immediately fired exactly one

2) I followed the instructions and removed the per result action. Instead I chose Number of events greater than 0. There was then a window to schedule in. When I went to save I was told "windowed real-time per result alerts require field based alert throttling to be enabled" . I attempted to do that and was told that I had to fill in "supress results containing field value". I was lost.

In summary, I think it is way more obvious what is happening to not use a real-time alert. Schedule your alert to run perhaps every minute for maybe the last -2m to last -1m (giving yourself plenty of time to index the event.) The expected behavior is far more understandable.

View solution in original post

SplunkTrust
SplunkTrust

Hi. So I have always avoid real-time alerts because I understand scheduled alerts. It seems the advice is that if you need something to be run frequently, run every minute.

When I look at real-time alerts (I am on 6.6.8: https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/DefineRealTimeAlerts) there are two ways to go.

I created an alert with the per-result triggering and made the action be to add to list of triggered alerts (I didn't want to spam myself with email)

1) Create a real-time alert with per-result triggering
I created one of these and could see, as admin, that the alert had a cron schedule of * * * * * (i.e. every minute) but when I caused the search to match and I looked at the triggered alerts, it immediately fired exactly one

2) I followed the instructions and removed the per result action. Instead I chose Number of events greater than 0. There was then a window to schedule in. When I went to save I was told "windowed real-time per result alerts require field based alert throttling to be enabled" . I attempted to do that and was told that I had to fill in "supress results containing field value". I was lost.

In summary, I think it is way more obvious what is happening to not use a real-time alert. Schedule your alert to run perhaps every minute for maybe the last -2m to last -1m (giving yourself plenty of time to index the event.) The expected behavior is far more understandable.

View solution in original post

New Member

Perhaps scheduling alerts would be the way to go. We don't have to know the second someone adds an account to domain admins, we just have to know in a reasonable time so we can verify if this action has been approved. Scheduling an alert once a minute should suffice. Can you explain to me how to accomplish scheduled alerts for what I am trying to do?

Thank you so much for your help with this.

Kenneth

0 Karma

SplunkTrust
SplunkTrust

Kenneth, you want to schedule an alert on a cron schedule:
https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/AlertSchedulingBestPractices

Earliest -2m
Latest -1m
Cron schedule would be * * * * *
That's every minute. You might only want to run the alert every 5 minutes. In which case you would

Earliest -6m
Latest -1m
Cron schedule: */5 * * * * which means run the alert every 5 minutes

Your search would be something that would include the codes you care about including the index
e.g. index= sourcetype= 4728 OR 4755

If this helps be sure to accept my answer. Thanks.

0 Karma

SplunkTrust
SplunkTrust

What time period are you searching over? You are looking for any events that match those codes in what time period? Do the emails stop?

0 Karma

New Member

Hello Burwell,

It is a real-time configuration. As far as I know, the Splunk system monitors the logs and at the very time it receives a log with an EventCode of 4728 or 4755, it triggers an email. This process is instantaneous and not configured for a time period. Did I answer your question?

alt text

0 Karma

SplunkTrust
SplunkTrust

I commented in my answer above. I think real-time alerts are too confusing.

0 Karma