Hi Fellow Splunkers,
I have an issue with triggered alerts failing to send email with authentication error (I use smtp). I found out that alert_actions.conf was mysteriously created under SPLUNK_HOME/etc/apps/[appname]/local/ with below stanza:
[email]
auth_password = encrypted value
This value takes precedence over system/local/alert_actions.conf and is the main reason why emails are not getting sent. These issues only come in my custom apps, search app is working as it should.
This is a splunk cluster with 3 search heads, this issue is seen in every search head and local/alert_actions.conf is always automatically created even after I deleted them. Note that I always push changes from master node, so I can't explain why is there a file in my custom apps under /local/ directories.
Any input would be appreciated, thank you!
