Alerting

Alert emails via Office 365

benhooper
Communicator

As per https://docs.microsoft.com/en-us/exchange/mail-flow-best-practices/how-to-set-up-a-multifunction-dev... ,  my email server settings are as follows:

  • Mail host: smtp.office365.com:587 (I've also tried using port 25 or omitting it entirely)
  • Email security: Enable TLS
  • Username: UPN of an Exchange Online-licensed 365 account.
  • Password: Password of the same Exchange Online-licensed 365 account.

2020 ∕ 10 ∕ 21 11꞉28꞉54 - Settings__Splunk_-_Google_Chrome.png

I created an alert normally and configured it to send an email:

2020 ∕ 10 ∕ 21 11꞉30꞉07 - Searches,_reports,_and_alerts__Splunk_8.0.6_-_Goo.png

However, the log reports two problems (probably related):

  1. It's trying to use server="localhost".
  2. SMTP AUTH extension not supported by server

2020 ∕ 10 ∕ 21 11꞉35꞉55 - Search__Splunk_8.0.6_-_Google_Chrome.png

The advanced settings of the alert show that it is aware of the 365 SMTP server.

From https://community.splunk.com/t5/Alerting/How-to-get-Office-365-integrated-as-my-SMTP-server-for-Splu... ,  I know it's normal to have to specify the server when using sendmail in a query and the advanced setting action.email.command uses sendmail but doesn't reference the server so I appended  server=$action.email.mailserver$ and even explicitly server=smtp.office365.com:587 but neither made a difference.

Can anyone help?

Labels (2)
0 Karma

apietersen
Communicator

After trying up/down, left/right, no succes sofar.

0 Karma

apietersen
Communicator

Splunk support suggest the following:

I was reviewing the know issues list and I found the issue number SPL-138647, See the link below for more information.

https://docs.splunk.com/Documentation/Splunk/8.1.2/ReleaseNotes/KnownIssues

0 Karma

apietersen
Communicator

exactly the same error in lines in pyhton.log

2021-03-03
13:26:49,738 +0100 ERROR sendemail:1600 - 'action.email.allowedDomainList' Traceback (most recent call last): File "D:\Program Files\Splunk\etc\apps\search\bin\sendemail.py", line 1593, in <module> results = sendEmail(results, settings, keywords, argvals) File "D:\Program Files\Splunk\etc\apps\search\bin\sendemail.py", line 469, in sendEmail

 

0 Karma

apietersen
Communicator

I have the same issue, from version 7.3 , with every new version (now 8.1.2)  I hope this would be solved.
Sendemail works only when you set the "Admin_All_Object" capability, but that is in fact a security breach for the regular user 

0 Karma

benhooper
Communicator

For testing, I set up the following new environment:

  • OS: Ubuntu Server 20.04
  • Splunk: Enterprise (full 2-month trial) 8.1.0
  • Alert: Exactly the same (installed as part of TA app)
  • Email server: smtp.office365.com:587
  • Email account credentials: Exactly the same

It works absolutely fine.

So, are all these problems a hidden limitation of the dev / test license?

0 Karma

benhooper
Communicator

As my dev / test instance of Splunk runs in a VM, I restored to the v8.0.6 snapshot, updated the SMTP credentials, and found that the problem and errors were unchanged.

0 Karma

benhooper
Communicator

As part of https://community.splunk.com/t5/forums/replypage/board-id/splunk-alerting/message-id/9866 , I upgraded the installation of Splunk Enterprise from v8.0.6 to v8.1.0 (latest as of writing). Due to the new feature Allowed Email Domains, this changed the error to the following:

 

2020-10-21 15:09:37,597 +0100 ERROR	sendemail:1599 - 'action.email.allowedDomainList'
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/search/bin/sendemail.py", line 1592, in <module>
    results = sendEmail(results, settings, keywords, argvals)
  File "/opt/splunk/etc/apps/search/bin/sendemail.py", line 469, in sendEmail
    ssContent['action.email.allowedDomainList'] = ssContent['action.email.allowedDomainList'].strip()
KeyError: 'action.email.allowedDomainList'

 

2020 ∕ 10 ∕ 21 15꞉17꞉36 - Search__Splunk_8.1.0_-_Google_Chrome.pngWeirdly, /en-GB/manager/search/admin/alert_actions/email?action=edit → Allowed Domains is marked with an asterisk as if it's required (which the above behaviour would suggest) but says Leave empty for no restriction.

2020 ∕ 10 ∕ 21 15꞉25꞉33 - Settings__Splunk_-_Google_Chrome.png

As this is a new feature, I just put this down to a bug, added our recipient domain name anyway, and verified that it pushed to the advanced settings of the alert but doing so didn't make a difference - the same errors continued.

2020 ∕ 10 ∕ 21 15꞉48꞉30 - New_Incident__Splunk_8.1.0_-_Google_Chrome.png

In file /opt/splunk/etc/apps/search/bin/sendemail.py, I added the lines beginning with logFile in the following block of code:

 

    logFile.write("ssContent action.email.allowedDomainList before: {}\n".format(ssContent.get('action.email.allowedDomainList')))
    logFile.write("alertContent allowedDomainList before: {}\n".format(alertContent.get('allowedDomainList')))
    ssContent['action.email.allowedDomainList'] = ssContent['action.email.allowedDomainList'].strip()
    logFile.write("ssContent action.email.allowedDomainList stripped: {}\n".format(ssContent.get('action.email.allowedDomainList')))
    if ssContent.get('action.email.allowedDomainList') != alertContent.get('allowedDomainList'):
        ssContent['action.email.allowedDomainList'] = alertContent['allowedDomainList']
        logger.warn("For alert=%s, the 'allowedDomainList' value is always obtained from alert_actions.conf."
                    "The allowedDomainList=%s" % (ssname, alertContent.get('allowedDomainList')))

    if alertContent.get('allowedDomainList') != "":
        if ssContent.get('action.email.mailserver') != alertContent.get('mailserver'):
            ssContent['action.email.mailserver'] = alertContent['mailserver']
            logger.warn("For alert=%s, if a 'allowedDomainList' is specified, it uses the 'mailserver'=%s in alert_actions.conf." %
                        (ssname, ssContent.get('action.email.mailserver')))

    logFile.write("action.email.allowedDomainList after: {}\n".format(ssContent.get('action.email.allowedDomainList')))
    logFile.write("alertContent allowedDomainList after: {}\n".format(alertContent.get('allowedDomainList')))

 

The log file contains the following:

 

ssContent action.email.allowedDomainList before: None
alertContent allowedDomainList before: <my domain>
ssContent action.email.allowedDomainList before: None
alertContent allowedDomainList before: <my domain>
ssContent action.email.allowedDomainList before: None
alertContent allowedDomainList before: <my domain>

 

So, given that it's not even getting passed .strip(), I can only assume that this is a bug.

Also, this 365 account was provisioned for me and I trust the person who did so but I thought I should check that the credentials are valid and found that the password was required to be changed. So, I've set a new password, verified access, and updated Splunk. However, doing so didn't make a difference either.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!