Splunk Add-on for Amazon Web Services: How to resolve error "UnicodeDecodeError: 'utf8' codec can't decode byte 0xd1 in position 0: invalid continuation byte"?

[SNaikwade]

Hi,

I am trying to onboard S3 data into AWS Cloud. I am using Splunk Add on for AWS on heavy forwarder. I have added the input for S3. This is the error I am getting.

04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py" UnicodeDecodeError: 'utf8' codec can't decode byte 0xd1 in position 0: invalid continuation byte

Please let me know in case I am missing on something.

Below is full error snippet from Logs:

04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py" Process Process-1:
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py" Traceback (most recent call last):
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     self.run()
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/multiprocessing/process.py", line 114, in run
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     self._target(*self._args, **self._kwargs)
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunksdc/collector.py", line 135, in _collector_work_procedure
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     collector.perform(name, params)
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunksdc/collector.py", line 234, in perform
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     self._delegate.perform(self, portal, checkpoint, name, params)
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py", line 345, in perform
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     self._handler.perform(app, portal, checkpoint, name, params)
04-03-2017 19:07:11.206 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3logs/handler.py", line 64, in perform
04-03-2017 19:07:11.207 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     if pipeline.run(portal, checkpoint):
04-03-2017 19:07:11.207 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunksdc/multitasking.py", line 151, in run
04-03-2017 19:07:11.207 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     self._delegate.done(portal, checkpoint, job, result)
04-03-2017 19:07:11.207 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3logs/adapter.py", line 167, in done
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     self._index_records(portal, job, result)
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/s3logs/adapter.py", line 212, in _index_records
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     portal.write(stream)
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunksdc/event.py", line 11, in write
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     text = stream.render()
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunksdc/event.py", line 54, in render
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     return ET.tostring(self._root)
04-03-2017 19:07:11.208 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/xml/etree/ElementTree.py", line 1126, in tostring
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     ElementTree(element).write(file, encoding, method=method)
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/xml/etree/ElementTree.py", line 820, in write
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     serialize(write, self._root, encoding, qnames, namespaces)
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/xml/etree/ElementTree.py", line 939, in _serialize_xml
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     _serialize_xml(write, e, encoding, qnames, None)
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/xml/etree/ElementTree.py", line 939, in _serialize_xml
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     _serialize_xml(write, e, encoding, qnames, None)
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/xml/etree/ElementTree.py", line 937, in _serialize_xml
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     write(_escape_cdata(text, encoding))
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"   File "/opt/splunk/lib/python2.7/xml/etree/ElementTree.py", line 1073, in _escape_cdata
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py"     return text.encode(encoding, "xmlcharrefreplace")
04-03-2017 19:07:11.209 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws_logs.py" UnicodeDecodeError: 'utf8' codec can't decode byte 0xd1 in position 0: invalid continuation byte

[dkuk]

Hi, Did you ever get to the bottom of this? I have the same issue.