Failed to send email due to encoding error

[tmontney]

From the splunkd.log:

05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -  Traceback (most recent call last):
05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -    File "C:\Program Files\Splunk\etc\apps\sendresults\bin\sendresults_alert.py", line 305, in <module>
05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -      sendemail(recipient, bcc, recipient_list[recipient].get('email_subj') , outbound, argvals)
05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -    File "C:\Program Files\Splunk\etc\apps\sendresults\bin\sendresults_alert.py", line 145, in sendemail
05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -      smtp.sendmail(sender, all_recipients, message)
05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -    File "C:\Program Files\Splunk\Python-3.7\lib\smtplib.py", line 855, in sendmail
05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -      msg = _fix_eols(msg).encode('ascii')
05-19-2020 16:36:03.808 -0500 ERROR sendmodalert - action=sendresults_alert STDERR -  UnicodeEncodeError: 'ascii' codec can't encode character '\u2003' in position 2239: ordinal not in range(128)
05-19-2020 16:36:04.317 -0500 INFO  sendmodalert - action=sendresults_alert - Alert action script completed in duration=2559 ms with exit code=1
05-19-2020 16:36:04.317 -0500 WARN  sendmodalert - action=sendresults_alert - Alert action script returned error code=1
05-19-2020 16:36:04.317 -0500 ERROR sendmodalert - Error in 'sendalert' command: Alert script returned error code 1.

Position in question was </td>. It was in the middle of the file, many came before and many came after. File was saved as UTF-8. Tried re-adding the tag, no effect. Tried minimizing the file, no effect. Eventually, I saved the file as ANSI and that did the trick. I do notice that the failure is coming from a Python library.

Sendresults is 5.0.0, on Splunk 8.0.2.1.

[andygerberkp]

I realize this is an old thread; but I just hit the same problem with sendresults, splunk 8.1.5, sendresults 5.0.3.

I was using a long comma separated list of email addresses in email_to.

Saw errors like:

" error="'ascii' codec can't encode character '\u202c' in position 29: ordinal not in range(128)"

The fix was to put a semicolon at the end of the email_to value.

So 

foo@bar.com,joe@foo.com,andy@whatever.org;

[PickleRick]

You might have unknowingly "fixed" the issue not by adding the semicolon itself but by re-reading/resubmitting/whatever the setting.

Typically those "hidden" unicode errors appear when you're copy-pasting some text which looks ascii but isn't (typical problem for me was where I wrote a documentation for some project and MS Word kept "fixing" quotes in the code; same often happens with dashes or spaces). \u202c is a character using in isolating parts of a text in context of text direction https://en.m.wikipedia.org/wiki/Bidirectional_text#Pops

[andygerberkp]

You were absolutely right; unprintable characters.  cat -v of the savedsearches.conf file revealed the problem, and a few tries in the GUI I was able to reset the search to remove the characters.  Thanks!

[nickbonz]

Try setting the system default encoding as utf-8 at the start of the script, so that all strings are encoded using that.

Example -

import sys
reload(sys)
sys.setdefaultencoding('utf-8')

The above should set the default encoding as utf-8 .

In most cases, the issue is that when you call str(), python uses the default character encoding to try and encode the bytes you gave it, which in your case are sometimes representations of unicode characters. To fix the problem, you have to tell python how to deal with the string you give it by using .encode('whatever_unicode'). Most of the time, you should be fine using utf-8. So, stop using str() to convert from unicode to encoded text / bytes, properly use .encode() to encode the string:,

yourstring.encode('utf-8')

For python 3.x ,there is default encoding, hence there will be no issue of encoding.

[PavelP]

Hello @tmontney,

UnicodeEncodeError: 'ascii' codec can't encode character '\u2003' indicates an unicode character cannot be ascii-encoded. The u2003 character is an 'EM SPACE' : https://www.fileformat.info/info/unicode/char/2003/index.htm

Try to find where/how it gets in there.

[mockd]

Hi,

Can you provide some more context on the issue? It looks like you modified the alert action script based on what I see in line 7 of the code snippet? Does this happen on every use of sendresults?

Thanks.

[tmontney]

Context is I'm testing a simple report with sendresults. Sender, subject, and message body filled out. Everything else empty or unchecked. My only other one is an alert and it's working. I used it as a template for this report.

[mockd]

Ah, ok. From that snippet it looked like line 7 was "new" but that might be from the python libraries. When you said you saved the file as ANSI it worked? When I was troubleshooting the other issue I had tested on windows with no issues.

Is there a chance there's some odd unicode character in the sender/subject/body fields of the alert action?

[tmontney]

As a matter of fact, there should've been: https://www.compart.com/en/unicode/U+03A9 However, when it was indexed by Splunk, it was converted to a "?". (Not sure if that was Splunk's doing or the script that sent the data.) To troubleshoot, I removed the HTML and the email sent OK.

[tmontney]

The only thing I've modified was as instructed here, by you: https://answers.splunk.com/answers/821371/sendresults-not-working-after-upgrade.html