Alerting

How to throttle or Suppress email alerts for multiple rows in result

sutom
Path Finder

Hello Everyone,

I am new to this place and this is my first query, looking for your help.

I have a use-case where I am trying to set an alert and make it dynamic based on the SLP query result, my recipient list is constant. but Alert is not working as I expected. I went through a lot of links and Splunk docs but still, I am in middle.

My requirement is to send the alert for every row from the result based on status and src(host IP) but I am receiving an alert only for the first row from the result.

Here is the query -

 

index=dummy uri_path
| stats count(eval(status>399)) as Error_Count by uri_path, status,user_name, src | where Error_Count > 0

 

Result -

uri_pathstatususer_namesrcError_count
/user/new400XXX123.21.321.121
/user/show404YYY321.12.32.211

My Alert Subject -

 

$result.status$ Error while access API for User $result.user_name$

 

My Message -

 

$result.status$ Error got observed while access API $result.uri_path$ with user $result.user_name$ on host $result.src$.
For more info please click on below link

 

My alert subject and message is getting update based on the result but I am constantly getting Alert for first row from result  - Splunk Alert: 400 Error while access API for User XXX. which is correct for first row

Some configuration in alert -

Alert type - Crone sachedule for 15 minutes,

Cron Expression - */15 * * * * , Expire - 24 hour

Trigger alert when - is greater then 0, Trigger - for each result.

Throttle - yes

Suppress results containing field value - src=$result.src$,

Suppress triggering for - 20-minutes

Still I am getting alert for first row from result,Not sure what I am missing here to get other rows alerts. If you can see I have suppressed based on src and in result SRC is different for both the rows. so based on this I should get both alerts but I am not.

Can anyone please help me to understand this, I want to send the alert based on status and src, if any new status + src combination come in result then it should send the result wether it is on first row in result or sencond row in result. 

Hope I am able to express my query.

 

Labels (3)
0 Karma
1 Solution

hoaxm3
Path Finder

I think it might be your suppression. You are saying when the src=$result.src$. Maybe try only suppressing off of "src" as the suppression will suppress the value for the specified field, you would not need to specify the value of suppression. Suppression = src. 

View solution in original post

0 Karma

hoaxm3
Path Finder

I think it might be your suppression. You are saying when the src=$result.src$. Maybe try only suppressing off of "src" as the suppression will suppress the value for the specified field, you would not need to specify the value of suppression. Suppression = src. 

0 Karma

sutom
Path Finder

Thanks @hoaxm3 it worked out, Now I am able to Suppression = src,uri_path,status with three field and getting result as expected.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...