Alert emails via Office 365

[benhooper]

As per https://docs.microsoft.com/en-us/exchange/mail-flow-best-practices/how-to-set-up-a-multifunction-dev... ,  my email server settings are as follows:

• Mail host: smtp.office365.com:587 (I've also tried using port 25 or omitting it entirely)
• Email security: Enable TLS
• Username: UPN of an Exchange Online-licensed 365 account.
• Password: Password of the same Exchange Online-licensed 365 account.

I created an alert normally and configured it to send an email:

However, the log reports two problems (probably related):

1. It's trying to use server="localhost".
2. SMTP AUTH extension not supported by server. 

The advanced settings of the alert show that it is aware of the 365 SMTP server.

From https://community.splunk.com/t5/Alerting/How-to-get-Office-365-integrated-as-my-SMTP-server-for-Splu... ,  I know it's normal to have to specify the server when using sendmail in a query and the advanced setting action.email.command uses sendmail but doesn't reference the server so I appended  server=$action.email.mailserver$ and even explicitly server=smtp.office365.com:587 but neither made a difference.

Can anyone help?

[apietersen]

After trying up/down, left/right, no succes sofar.

[apietersen]

Splunk support suggest the following:

I was reviewing the know issues list and I found the issue number SPL-138647, See the link below for more information.

https://docs.splunk.com/Documentation/Splunk/8.1.2/ReleaseNotes/KnownIssues

[apietersen]

exactly the same error in lines in pyhton.log

2021-03-03 13:26:49,738 +0100 ERROR sendemail:1600 - 'action.email.allowedDomainList' Traceback (most recent call last): File "D:\Program Files\Splunk\etc\apps\search\bin\sendemail.py", line 1593, in <module> results = sendEmail(results, settings, keywords, argvals) File "D:\Program Files\Splunk\etc\apps\search\bin\sendemail.py", line 469, in sendEmail

[apietersen]

I have the same issue, from version 7.3 , with every new version (now 8.1.2)  I hope this would be solved.
Sendemail works only when you set the "Admin_All_Object" capability, but that is in fact a security breach for the regular user 

[benhooper]

For testing, I set up the following new environment:

• OS: Ubuntu Server 20.04
• Splunk: Enterprise (full 2-month trial) 8.1.0
• Alert: Exactly the same (installed as part of TA app)
• Email server: smtp.office365.com:587
• Email account credentials: Exactly the same

It works absolutely fine.

So, are all these problems a hidden limitation of the dev / test license?

[benhooper]

As my dev / test instance of Splunk runs in a VM, I restored to the v8.0.6 snapshot, updated the SMTP credentials, and found that the problem and errors were unchanged.

[benhooper]

As part of https://community.splunk.com/t5/forums/replypage/board-id/splunk-alerting/message-id/9866 , I upgraded the installation of Splunk Enterprise from v8.0.6 to v8.1.0 (latest as of writing). Due to the new feature Allowed Email Domains, this changed the error to the following:

2020-10-21 15:09:37,597 +0100 ERROR	sendemail:1599 - 'action.email.allowedDomainList'
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/search/bin/sendemail.py", line 1592, in <module>
    results = sendEmail(results, settings, keywords, argvals)
  File "/opt/splunk/etc/apps/search/bin/sendemail.py", line 469, in sendEmail
    ssContent['action.email.allowedDomainList'] = ssContent['action.email.allowedDomainList'].strip()
KeyError: 'action.email.allowedDomainList'

Weirdly, /en-GB/manager/search/admin/alert_actions/email?action=edit → Allowed Domains is marked with an asterisk as if it's required (which the above behaviour would suggest) but says Leave empty for no restriction.

As this is a new feature, I just put this down to a bug, added our recipient domain name anyway, and verified that it pushed to the advanced settings of the alert but doing so didn't make a difference - the same errors continued.

In file /opt/splunk/etc/apps/search/bin/sendemail.py, I added the lines beginning with logFile in the following block of code:

    logFile.write("ssContent action.email.allowedDomainList before: {}\n".format(ssContent.get('action.email.allowedDomainList')))
    logFile.write("alertContent allowedDomainList before: {}\n".format(alertContent.get('allowedDomainList')))
    ssContent['action.email.allowedDomainList'] = ssContent['action.email.allowedDomainList'].strip()
    logFile.write("ssContent action.email.allowedDomainList stripped: {}\n".format(ssContent.get('action.email.allowedDomainList')))
    if ssContent.get('action.email.allowedDomainList') != alertContent.get('allowedDomainList'):
        ssContent['action.email.allowedDomainList'] = alertContent['allowedDomainList']
        logger.warn("For alert=%s, the 'allowedDomainList' value is always obtained from alert_actions.conf."
                    "The allowedDomainList=%s" % (ssname, alertContent.get('allowedDomainList')))

    if alertContent.get('allowedDomainList') != "":
        if ssContent.get('action.email.mailserver') != alertContent.get('mailserver'):
            ssContent['action.email.mailserver'] = alertContent['mailserver']
            logger.warn("For alert=%s, if a 'allowedDomainList' is specified, it uses the 'mailserver'=%s in alert_actions.conf." %
                        (ssname, ssContent.get('action.email.mailserver')))

    logFile.write("action.email.allowedDomainList after: {}\n".format(ssContent.get('action.email.allowedDomainList')))
    logFile.write("alertContent allowedDomainList after: {}\n".format(alertContent.get('allowedDomainList')))

The log file contains the following:

ssContent action.email.allowedDomainList before: None
alertContent allowedDomainList before: <my domain>
ssContent action.email.allowedDomainList before: None
alertContent allowedDomainList before: <my domain>
ssContent action.email.allowedDomainList before: None
alertContent allowedDomainList before: <my domain>

So, given that it's not even getting passed .strip(), I can only assume that this is a bug.

Also, this 365 account was provisioned for me and I trust the person who did so but I thought I should check that the credentials are valid and found that the password was required to be changed. So, I've set a new password, verified access, and updated Splunk. However, doing so didn't make a difference either.