I'm adding a comment here for anyone else encountering this issue, which is apparently going to be fixed in 6.5.3. Note this is assuming you do not use ssl or tls when sending email.
I applied the fixes to my server.conf but the error above continued to occur, even despite working with support to try to identify the problem. After getting all the way to the development level, we discovered that we needed to force Splunk to run a successful search with email without ssl or tls enabled before any other emails could succeed. You can do this from the GUI by making the changes, restarting the Splunk service, and running a search like the following:
head 100 | top 5 host | sendemail to="youremailaddress@whatever.com" use_ssl=0 use_tls=0
If you get email, re-run this search without the use_ssl or use_tls parameters and verify you still get it.
The developer theorized that due to both the bug of not expanding the variable and how the sendemail.py script works in 6.5.x (it still initializes SSL context even if you don't use SSL, in preparation for if your use_ssl is set to enabled), that if it fails, it will have a remnant of that failure in opening the cert file and subsequently fail every other attempt to send email. By forcing a successful sending of mail in bypassing ssl and tls, you basically complete the mail process and clear anything that was cached or stored from before, and email alerts succeed once again.
So if you try this solution and it still doesn't work, try it with the search I've indicated and see if that clears the cert open failure.
... View more
