Splunk Enterprise Security: How do you stop emails from a suppressed alert in a correlation search?

[theslobb]

I have a search that monitors alerts created by an IDS. I have begun going through the triggered alerts to suppress the known false positives, however, I still receive an email notification about the suppressed alert after it is triggered in the correlation search.

How do I turn off the emails for suppressed alerts?

[dylan_yoder]

I figured out a workaround to this. By adding the following to the end of my correlation rule it checks the suppressed eventtypes first to see if there are anything that is suppressed with matching fields. If it find anything with matching fields that hasn't expired it doesn't return any results so the emails/tickets/notable/adaptive response actions are not triggered.

| search NOT
[ | 'suppression_eventtypes '
| eval _raw = search
| extract
| search source = "insert correlation rule name here" | eval tnow = now() | where tnow > start_time AND tnow ]

suppression _eventtypes is a macro..

| rest splunk_server=local count=0 /services/saved/eventtypes
| search title=notable_suppression*
| rename title as eventtype
| rex field=eventtype "notable_suppression-(?.+)"
| rex field=search "_time>=?(?\d+)"
| rex field=search "_time<=?(?\d+)"

[rdeloach]

The answer from Splunk is that the notable event suppression only hides notable events from the Incident Review dashboard. SInce the alert conditions are still met it will still fire the Adaptive Response action, send email, etc. The only way to prevent the alert from firing any other action is to either build the suppression in your correlation search or change the alert trigger conditions.

[wilhelmF]

It would be good if splunk could provide a template how to dynamically at supressions when supressed in the inident review.

[wilhelmF]

@rdeloach Did you get an answer from Splunk?

[dacosta]

TheSlobb, did you ever get an answer on this? Im getting emails on a suppressed alert as well.

thanks,

Dan

[wilhelmF]

No I still don't have suitable workaround.

[rdeloach]

I'm having this issue also. Not only with emails but adaptive response actions also. Trying to suppress a notable event from occurring but still getting a barrage of emails or incidents in a ticketing system isn't ideal. I'm going to put a support ticket in to see if they have any answers on it.

[dylan_yoder]

I figured out a workaround to this. By adding the following to the end of my correlation rule it checks the suppressed eventtypes first to see if there are anything that is suppressed with matching fields. If it find anything with matching fields that hasn't expired it doesn't return any results so the emails/tickets/notable/adaptive response actions are not triggered.

| search NOT
[ | suppression_eventtypes
| eval _raw = search
| extract
| search source = ""
| eval tnow = now()
| where tnow > start_time AND tnow
| table ]

[dylan_yoder]

suppression _eventtypes is a macro..

| rest splunk_server=local count=0 /services/saved/eventtypes
| search title=notable_suppression*
| rename title as eventtype
| rex field=eventtype "notable_suppression-(?.+)"
| rex field=search "_time>=?(?\d+)"
| rex field=search "_time<=?(?\d+)"

[wilhelmF]

Meanwhile I created an additional correlation search which sends a mail. It's not optimal but it works.

| inputlookup es_notable_events 
 | search urgency=high OR urgency=critical
 | eval last5min=now()-310
 | where _time >= last5min

[theslobb]

This is clever, and I really wish I could use this as an alternative. However, the information in the notable events logs are not enough to give context to the users who receive the alerts most times.

[phillip_rice]

Was this ever resolved?

Thanks

[theslobb]

I never got a resolution for this one.