Alerting

How to stop a single account email alert to trigger multiple emails?

k45bryant
New Member

I configured an alert to send an email every time a user is added to the Domain Admins group. I have this alert triggering on eventcode 4728, 4755, etc. The problem is that adding a single account will trigger multiple emails. I want the first event to trigger an email, but all subsequent events not to trigger an email. How do I accomplish this?

0 Karma
1 Solution

burwell
SplunkTrust
SplunkTrust

Hi. So I have always avoid real-time alerts because I understand scheduled alerts. It seems the advice is that if you need something to be run frequently, run every minute.

When I look at real-time alerts (I am on 6.6.8: https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/DefineRealTimeAlerts) there are two ways to go.

I created an alert with the per-result triggering and made the action be to add to list of triggered alerts (I didn't want to spam myself with email)

1) Create a real-time alert with per-result triggering
I created one of these and could see, as admin, that the alert had a cron schedule of * * * * * (i.e. every minute) but when I caused the search to match and I looked at the triggered alerts, it immediately fired exactly one

2) I followed the instructions and removed the per result action. Instead I chose Number of events greater than 0. There was then a window to schedule in. When I went to save I was told "windowed real-time per result alerts require field based alert throttling to be enabled" . I attempted to do that and was told that I had to fill in "supress results containing field value". I was lost.

In summary, I think it is way more obvious what is happening to not use a real-time alert. Schedule your alert to run perhaps every minute for maybe the last -2m to last -1m (giving yourself plenty of time to index the event.) The expected behavior is far more understandable.

View solution in original post

burwell
SplunkTrust
SplunkTrust

Hi. So I have always avoid real-time alerts because I understand scheduled alerts. It seems the advice is that if you need something to be run frequently, run every minute.

When I look at real-time alerts (I am on 6.6.8: https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/DefineRealTimeAlerts) there are two ways to go.

I created an alert with the per-result triggering and made the action be to add to list of triggered alerts (I didn't want to spam myself with email)

1) Create a real-time alert with per-result triggering
I created one of these and could see, as admin, that the alert had a cron schedule of * * * * * (i.e. every minute) but when I caused the search to match and I looked at the triggered alerts, it immediately fired exactly one

2) I followed the instructions and removed the per result action. Instead I chose Number of events greater than 0. There was then a window to schedule in. When I went to save I was told "windowed real-time per result alerts require field based alert throttling to be enabled" . I attempted to do that and was told that I had to fill in "supress results containing field value". I was lost.

In summary, I think it is way more obvious what is happening to not use a real-time alert. Schedule your alert to run perhaps every minute for maybe the last -2m to last -1m (giving yourself plenty of time to index the event.) The expected behavior is far more understandable.

k45bryant
New Member

Perhaps scheduling alerts would be the way to go. We don't have to know the second someone adds an account to domain admins, we just have to know in a reasonable time so we can verify if this action has been approved. Scheduling an alert once a minute should suffice. Can you explain to me how to accomplish scheduled alerts for what I am trying to do?

Thank you so much for your help with this.

Kenneth

0 Karma

burwell
SplunkTrust
SplunkTrust

Kenneth, you want to schedule an alert on a cron schedule:
https://docs.splunk.com/Documentation/Splunk/6.6.8/Alert/AlertSchedulingBestPractices

Earliest -2m
Latest -1m
Cron schedule would be * * * * *
That's every minute. You might only want to run the alert every 5 minutes. In which case you would

Earliest -6m
Latest -1m
Cron schedule: */5 * * * * which means run the alert every 5 minutes

Your search would be something that would include the codes you care about including the index
e.g. index= sourcetype= 4728 OR 4755

If this helps be sure to accept my answer. Thanks.

0 Karma

burwell
SplunkTrust
SplunkTrust

What time period are you searching over? You are looking for any events that match those codes in what time period? Do the emails stop?

0 Karma

k45bryant
New Member

Hello Burwell,

It is a real-time configuration. As far as I know, the Splunk system monitors the logs and at the very time it receives a log with an EventCode of 4728 or 4755, it triggers an email. This process is instantaneous and not configured for a time period. Did I answer your question?

alt text

0 Karma

burwell
SplunkTrust
SplunkTrust

I commented in my answer above. I think real-time alerts are too confusing.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...