Alerting

How to throttle or Suppress email alerts for multiple rows in result

sutom
Engager

Hello Everyone,

I am new to this place and this is my first query, looking for your help.

I have a use-case where I am trying to set an alert and make it dynamic based on the SLP query result, my recipient list is constant. but Alert is not working as I expected. I went through a lot of links and Splunk docs but still, I am in middle.

My requirement is to send the alert for every row from the result based on status and src(host IP) but I am receiving an alert only for the first row from the result.

Here is the query -

 

index=dummy uri_path
| stats count(eval(status>399)) as Error_Count by uri_path, status,user_name, src | where Error_Count > 0

 

Result -

uri_pathstatususer_namesrcError_count
/user/new400XXX123.21.321.121
/user/show404YYY321.12.32.211

My Alert Subject -

 

$result.status$ Error while access API for User $result.user_name$

 

My Message -

 

$result.status$ Error got observed while access API $result.uri_path$ with user $result.user_name$ on host $result.src$.
For more info please click on below link

 

My alert subject and message is getting update based on the result but I am constantly getting Alert for first row from result  - Splunk Alert: 400 Error while access API for User XXX. which is correct for first row

Some configuration in alert -

Alert type - Crone sachedule for 15 minutes,

Cron Expression - */15 * * * * , Expire - 24 hour

Trigger alert when - is greater then 0, Trigger - for each result.

Throttle - yes

Suppress results containing field value - src=$result.src$,

Suppress triggering for - 20-minutes

Still I am getting alert for first row from result,Not sure what I am missing here to get other rows alerts. If you can see I have suppressed based on src and in result SRC is different for both the rows. so based on this I should get both alerts but I am not.

Can anyone please help me to understand this, I want to send the alert based on status and src, if any new status + src combination come in result then it should send the result wether it is on first row in result or sencond row in result. 

Hope I am able to express my query.

 

Labels (3)
0 Karma
1 Solution

hoaxm3
Path Finder

I think it might be your suppression. You are saying when the src=$result.src$. Maybe try only suppressing off of "src" as the suppression will suppress the value for the specified field, you would not need to specify the value of suppression. Suppression = src. 

View solution in original post

0 Karma

hoaxm3
Path Finder

I think it might be your suppression. You are saying when the src=$result.src$. Maybe try only suppressing off of "src" as the suppression will suppress the value for the specified field, you would not need to specify the value of suppression. Suppression = src. 

View solution in original post

0 Karma

sutom
Engager

Thanks @hoaxm3 it worked out, Now I am able to Suppression = src,uri_path,status with three field and getting result as expected.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.