Hi,
I've recently installed the Alert Manager (and the add-on) on a search head cluster. I've added the Alert Manager trigger action to some saved searches and followed the instructions on how to configure email notification in Alert Manager. Using Splunk's email trigger action works for the same alert, but not when I try email notification using the Alert Manager. I see evidence in the alert_manager_notifications.log that it is trying to send a mail on the "incident_created" event:
/opt/splunk/var/log/splunk/alert_manager_notifications.log:
2019-03-07 14:05:07,174 INFO pid="141090" logger="alert_manager_notifications" message="Start trying to send notification to [u'xxxxx@xxxxx.xx'] with event=incident_created of alert test alert manager" (NotificationHandler.py:189)
Would be great if somebody out there have any ideas on how to troubleshoot this.
Update:
I see this error in the alert_manager_notifications.log:
2019-03-11 20:45:46,449 ERROR pid="80153" logger="alert_manager_notifications" message="Unable to send notification. Continuing without sending notification. Unexpected Error: Traceback (most recent call last):
File "/opt/splunk/etc/apps/alert_manager/bin/lib/NotificationHandler.py", line 200, in send_notification
content = template.render(context)
File "/opt/splunk/etc/apps/alert_manager/bin/lib/jinja2/environment.py", line 989, in render
return self.environment.handle_exception(exc_info, True)
File "/opt/splunk/etc/apps/alert_manager/bin/lib/jinja2/environment.py", line 754, in handle_exception
reraise(exc_type, exc_value, tb)
File "/opt/splunk/etc/apps/alert_manager/default/templates/default.html", line 52, in top-level template code
{% for k in results[0] %}
UndefinedError: 'results' is undefined
" (NotificationHandler.py:332)
