Reporting

Why are the email alerts not being sent anymore?

nick405060
Motivator

Alerts no longer email, however they do show up in triggered alerts. This started sometime yesterday, before that we have been getting email alerts and creating email alerts for months.

var\log\splunk\python.log shows:

2018-08-16 15:44:41,450 PDT ERROR sendemail:115 - Sending email. subject="testalert010", results_link="http://ABCSPLNK02:8000/app/abc_ist/search?q=%7Cloadjob%20rt_scheduler__username_ZHd0X2lzdA__testtest_at_1534459452_73.0%20%7C%20head%201%20%7C%20tail%201&earliest=0&latest=now", recipients="[u'myemailaddress@abc.com']", server="localhost" 
2018-08-16 15:44:41,450 PDT ERROR sendemail:378 - [Errno 10061] No connection could be made because the target machine actively refused it while sending mail to: u'myemailaddress@abc.com 

We've checked and there is no firewall/exchange/mimecast/smtp blocking the emails. I have been messing with limits.conf, savedsearches.conf, and alert_actions.conf for completely separate reasons, and those changes should not have affected email alerting. I went back and deleted every one of those mentioned .conf files that have been edited in the last three days (in the local directories, so the default files have taken back over) and still no luck.

I also installed the Slack app in the last three days, is it possible that the installation overrode SMTP settings in Splunk or something? (I have since deleted the Slack app in case that was the reason why, without it fixing the problem)

Have looked at all Splunk answers for this issue to no avail.

Tags (2)
1 Solution

nick405060
Motivator

Posted this question so I could answer it for the Splunk community. As explained by a Splunk support engineer:

My recommendation is to set the server back to "smtp.abc.com" by navigating in the GUI Settings->Server settings->Email settings 
Then change the "Mail host" field from localhost, to smtp.abc.com. 

Not sure how it got changed, and it's interesting that the change was not reflected in any config files anywhere, so hours of searching in PowerShell wasn't able to detect the change. My theory is that installing our Slack integration app defaulted the mail host back to localhost

View solution in original post

nick405060
Motivator

Posted this question so I could answer it for the Splunk community. As explained by a Splunk support engineer:

My recommendation is to set the server back to "smtp.abc.com" by navigating in the GUI Settings->Server settings->Email settings 
Then change the "Mail host" field from localhost, to smtp.abc.com. 

Not sure how it got changed, and it's interesting that the change was not reflected in any config files anywhere, so hours of searching in PowerShell wasn't able to detect the change. My theory is that installing our Slack integration app defaulted the mail host back to localhost

abhijitsaoji
Explorer

Hi, I am facing same issue, my real time alert is not working at all. It is neither appearing in the triggered alerts nor sending any emails. I have changed the alert type to scheduled - every hour on 30th minute and worked like a charm. Not sure what would be the issue with real time, I have read few comments about latency and ran the query supplied but latency is coming in seconds so probably it can be ruled out. any other thoughts, please let me know.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...