Alerting

Why does my real-time alert continue to send emails for events that happened hours ago?

di2esysadmin
Path Finder

I have a simple search:

host=*prod*  "Too many open files"  source!="/opt/atlassian/jira-data/log/emh.log*"

I've set up a simple alert. Real-time, throttling on host for 1 minute. (pic attached)

We had 200+ of these errors, all within 2 seconds of each other, from a single host 2 hours ago. Since then, we've received no less than 24 emails alerting us of those events. I want one email, not 24.

What have I misconfigured?

Thanks!

alt text

0 Karma
1 Solution

linu1988
Champion

Hi,
Please try to set the below setting with your preferred field. Always try to table to data you require. Moreover the throttling period is only 1 minute, so if you have events matching it will generate the alert. If you want hourly then throttle it to send 1 hour per host in the suppress triggering option.

alt text

Thanks,
L

View solution in original post

0 Karma

linu1988
Champion

Hi,
Please try to set the below setting with your preferred field. Always try to table to data you require. Moreover the throttling period is only 1 minute, so if you have events matching it will generate the alert. If you want hourly then throttle it to send 1 hour per host in the suppress triggering option.

alt text

Thanks,
L

0 Karma

di2esysadmin
Path Finder

Thank you. I've made these changes. I've my fingers crossed this will do the trick.

0 Karma
Get Updates on the Splunk Community!

Devesh Logendran, Splunk, and the Singapore Cyber Conquest

At this year’s Splunk University, I had the privilege of chatting with Devesh Logendran, one of the winners in ...

There's No Place Like Chrome and the Splunk Platform

WATCH NOW!Malware. Risky Extensions. Data Exfiltration. End-users are increasingly reliant on browsers to ...

Customer Experience | Join the Customer Advisory Board!

Are you ready to take your Splunk journey to the next level? 🚀 We invite you to join our elite squad ...