Reporting

Why are the email alerts not being sent anymore?

nick405060
Motivator

Alerts no longer email, however they do show up in triggered alerts. This started sometime yesterday, before that we have been getting email alerts and creating email alerts for months.

var\log\splunk\python.log shows:

2018-08-16 15:44:41,450 PDT ERROR sendemail:115 - Sending email. subject="testalert010", results_link="http://ABCSPLNK02:8000/app/abc_ist/search?q=%7Cloadjob%20rt_scheduler__username_ZHd0X2lzdA__testtest_at_1534459452_73.0%20%7C%20head%201%20%7C%20tail%201&earliest=0&latest=now", recipients="[u'myemailaddress@abc.com']", server="localhost" 
2018-08-16 15:44:41,450 PDT ERROR sendemail:378 - [Errno 10061] No connection could be made because the target machine actively refused it while sending mail to: u'myemailaddress@abc.com 

We've checked and there is no firewall/exchange/mimecast/smtp blocking the emails. I have been messing with limits.conf, savedsearches.conf, and alert_actions.conf for completely separate reasons, and those changes should not have affected email alerting. I went back and deleted every one of those mentioned .conf files that have been edited in the last three days (in the local directories, so the default files have taken back over) and still no luck.

I also installed the Slack app in the last three days, is it possible that the installation overrode SMTP settings in Splunk or something? (I have since deleted the Slack app in case that was the reason why, without it fixing the problem)

Have looked at all Splunk answers for this issue to no avail.

Tags (2)
1 Solution

nick405060
Motivator

Posted this question so I could answer it for the Splunk community. As explained by a Splunk support engineer:

My recommendation is to set the server back to "smtp.abc.com" by navigating in the GUI Settings->Server settings->Email settings 
Then change the "Mail host" field from localhost, to smtp.abc.com. 

Not sure how it got changed, and it's interesting that the change was not reflected in any config files anywhere, so hours of searching in PowerShell wasn't able to detect the change. My theory is that installing our Slack integration app defaulted the mail host back to localhost

View solution in original post

nick405060
Motivator

Posted this question so I could answer it for the Splunk community. As explained by a Splunk support engineer:

My recommendation is to set the server back to "smtp.abc.com" by navigating in the GUI Settings->Server settings->Email settings 
Then change the "Mail host" field from localhost, to smtp.abc.com. 

Not sure how it got changed, and it's interesting that the change was not reflected in any config files anywhere, so hours of searching in PowerShell wasn't able to detect the change. My theory is that installing our Slack integration app defaulted the mail host back to localhost

abhijitsaoji
Explorer

Hi, I am facing same issue, my real time alert is not working at all. It is neither appearing in the triggered alerts nor sending any emails. I have changed the alert type to scheduled - every hour on 30th minute and worked like a charm. Not sure what would be the issue with real time, I have read few comments about latency and ran the query supplied but latency is coming in seconds so probably it can be ruled out. any other thoughts, please let me know.

0 Karma
Get Updates on the Splunk Community!

Discover Powerful New Features in Splunk Cloud Platform: Enhanced Analytics, ...

Hey Splunky people! We are excited to share the latest updates in Splunk Cloud Platform 9.3.2408. In this ...

Splunk Classroom Chronicles: Training Tales and Testimonials

Welcome to the "Splunk Classroom Chronicles" series, created to help curious, career-minded learners get ...

Access Tokens Page - New & Improved

Splunk Observability Cloud recently launched an improved design for the access tokens page for better ...