All Apps and Add-ons
Highlighted

Alert manager - email notification

Engager

Hi,

I've recently installed the Alert Manager (and the add-on) on a search head cluster. I've added the Alert Manager trigger action to some saved searches and followed the instructions on how to configure email notification in Alert Manager. Using Splunk's email trigger action works for the same alert, but not when I try email notification using the Alert Manager. I see evidence in the alertmanagernotifications.log that it is trying to send a mail on the "incident_created" event:

/opt/splunk/var/log/splunk/alertmanagernotifications.log:

2019-03-07 14:05:07,174 INFO pid="141090" logger="alertmanagernotifications" message="Start trying to send notification to [u'xxxxx@xxxxx.xx'] with event=incident_created of alert test alert manager" (NotificationHandler.py:189)

Would be great if somebody out there have any ideas on how to troubleshoot this.

Update:

I see this error in the alertmanagernotifications.log:

2019-03-11 20:45:46,449 ERROR pid="80153" logger="alertmanagernotifications" message="Unable to send notification. Continuing without sending notification. Unexpected Error: Traceback (most recent call last):
File "/opt/splunk/etc/apps/alertmanager/bin/lib/NotificationHandler.py", line 200, in sendnotification
content = template.render(context)
File "/opt/splunk/etc/apps/alertmanager/bin/lib/jinja2/environment.py", line 989, in render
return self.environment.handle
exception(excinfo, True)
File "/opt/splunk/etc/apps/alert
manager/bin/lib/jinja2/environment.py", line 754, in handleexception
reraise(exc
type, excvalue, tb)
File "/opt/splunk/etc/apps/alert
manager/default/templates/default.html", line 52, in top-level template code
{% for k in results[0] %}
UndefinedError: 'results' is undefined
" (NotificationHandler.py:332)