Alerting

How to get one alert for all the same alerts?

cramery
New Member

I want to make an Alert, as soon as a Application gets startet (in this case Firefox). But for somehow, always 4 or more alerts get triggered. But the Alerts are exact the same to each other, theres not a single diffrent in the whole alert. How can I reach, that I only get one Alert for all of these same Alerts, that are also in the exact same time? (Also, when I just use it as a search, it perfectly works fine too)

My Search:

source="WinEventLog:Microsoft-Windows-Sysmon/Operational" (Image="C:\\Program Files\\Mozilla Firefox\\firefox.exe") | stats first(*) by ParentProcessId

My Alert conditions:

Real-Time
Number of Results: Equals 2 (It only works with 2, dont know why)
In 1 Minute
Triggers Once
Add to Triggered Alerts
Tags (2)
0 Karma

woodcock
Esteemed Legend

First of all, stop using realtime and your problem goes away; then search like this:

source="WinEventLog:Microsoft-Windows-Sysmon/Operational" (Image="C:\\Program Files\\Mozilla Firefox\\firefox.exe") | stats first(*) by ParentProcessId host
0 Karma

nickhills
Ultra Champion

This is the problem with Real-Time alerts.

Consider this:
you have a search which generates a result (like yours)
It is looking for events which have occurred in the last 60 seconds.

Your event fires at 05 seconds past the minute - the RT search runs, it fulfils the search criteria, so it generates an alert, and 'Triggers Once' - an email is sent etc, the 'Job' completes

It is now 15 seconds past the minute - your RT search runs again, the result set STILL fulfils the criteria, so it triggers 'Once' again..

etc, etc

Better is to schedule your searches to run every minute looking back at the previous 60 seconds. (if you really must) but this kind of search does not work well with RT alerts, You 'could' use throttling to stop the same alert firing more than once, but you run the risk that two similar (but different) events could trigger resulting in throttling a message you care about.

The 'look back' method is better to make sure you don't get multiple alerts firing for a single event, and that each event gets alerted.

An even better approach is to widen the search window further, and look for events earliest=-2m, latest=-1m - This helps protect against messages which have been delayed by a few seconds and might otherwise be missed.

Better again is to run it over a longer time, like every 15 mins search -16min to -1m, but the immediacy is impacted

If my comment helps, please give it a thumbs up!
0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...