Alerting

Need Help w/ Multiple Alert Emails

joeybroesky
Path Finder

Need assistance figuring out why we are receiving multiple email alerts. We are trying to setup email alerts for Office365 Service Messages.

Search String

index="o365data" sourcetype="o365:service:message"  Id=*
 | where Classification == "Incident" AND Severity == "Sev2"
 | spath Messages{} output=Messages
 | spath WorkloadDisplayName
 | spath Id
 | spath Status
 | stats values(WorkloadDisplayName) as WorkloadDisplayName values(Id) as Id values(Status) as Status by Messages
 | spath input=Messages
 | eval PublishedTime=strptime(PublishedTime, "%Y-%m-%dT%H:%M:%S.%NZ")
 | eval CorrectPublished=PublishedTime+25200
 | where MessageText != "A post-incident report has been published."
 | stats count by CorrectPublished Id WorkloadDisplayName MessageText Status 
 | sort - CorrectPublished
 | dedup CorrectPublished
 | fields - count
 | eval CorrectPublished=strftime(CorrectPublished,"%Y/%m/%d %T")
 | fields - PublishedTime
 | dedup Id
 | table CorrectPublished Id WorkloadDisplayName MessageText Status
 | rename CorrectPublished as "Published", Id as "ID", WorkloadDisplayName as "Workload", MessageText as "Details"

Alert Settings
Alert Type: Real-time
Expires: 48 Hour(s)
Trigger alert when: Per-Result
Throtte: yes
Suppress results containing field value: *
Suppress triggering for: 2 minute(s)
When triggered: Send email

Emails Received
alt text

alt text

Labels (2)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Try changing Trigger alert when: Per-Result to Trigger alert when: Once.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try changing Trigger alert when: Per-Result to Trigger alert when: Once.

---
If this reply helps you, Karma would be appreciated.
0 Karma

joeybroesky
Path Finder

Thanks Rich! I'll test this and report back!

0 Karma

joeybroesky
Path Finder

@richgalloway my apologies for the delay, I should be able to verify on the next alert.

0 Karma

joeybroesky
Path Finder

This did the trick. Thanks again Rich!!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

@joeybroesky If your problem is solved then please accept the answer to help future readers.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...