Splunk Add-on for Check Point OPSEC LEA 4.0.0: "UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 330: invalid continuation byte"

[kmuellercm]

R77 with dedicated logging server
Enabled a LEA connection and I get just a few logs, then the process bombs out. Logs from var/log/splunk/splunk_ta_checkpoint-opseclea_modinput.log indicate EventWriter encountered an exception, then stops processing inputs. I get about 150k log lines parsed properly. No obvious errors in splunkd.log and the loggrabber process remains running:

 /opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/../bin/lea_loggrabber --data non_audit --debug_level 2 --appname Splunk_TA_checkpoint-opseclea --lea_server_ip 1.2.3.4 --lea_server_auth_port 18184 --lea_server_auth_type sslca --opsec_sslca_file /opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/certs/SplunkLEA.p12 --opsec_sic_name CN=SplunkLEA--opsec_entity_sic_name CN=MGMTSVR --last_record_location 1471966084 115426 --online --no_resolve

var/log/splunk/splunk_ta_checkpoint-opseclea_modinput.log:

2016-08-23 16:13:16,654 +0000 log_level=INFO, pid=17396, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"]log_level=2 file:lea_loggrabber.cpp func_name:read_fw1_logfile_collogs code_line_no:2052 :LEA collected logfile handler was invoked
2016-08-23 16:13:27,679 +0000 log_level=ERROR, pid=17396, tid=Thread-1, file=event_writer.py, func_name=_do_write_events, code_line_no=79 | EventWriter encounter exception which maycause data loss, queue leftsize=3
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktalib/event_writer.py", line 62, in _do_write_events
    for evt in event:
  File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktaucclib/data_collection/ta_data_collector.py", line 59, in <genexpr>
    index, scu.escape_cdata(event.event)) for event
  File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktalib/common/util.py", line 71, in escape_cdata
    data = data.encode("utf-8", errors="xmlcharrefreplace")
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 330: invalid continuation byte
2016-08-23 16:13:27,679 +0000 log_level=INFO, pid=17396, tid=Thread-1, file=event_writer.py, func_name=_do_write_events, code_line_no=84 | Event writer stopped, queue leftsize=4
2016-08-23 16:13:27,680 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=ta_data_collector.py, func_name=_write_events, code_line_no=122 | [input_name="Checkpoint NonAudit Events" data="non_audit"]  the event queue is closed and the received data will be discarded
2016-08-23 16:13:27,681 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=ta_data_collector.py, func_name=index_data, code_line_no=114 | [input_name="Checkpoint NonAudit Events" data="non_audit"]  End of indexing data for Checkpoint NonAudit Events_non_audit
2016-08-23 16:13:27,681 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=thread_pool.py, func_name=_run, code_line_no=261 | Thread work_queue_size=0

[jamesarmitage · Accepted solution]

It looks like you're encountering the same issue I did. I made a workaround that requires modifying a single line in the TA:

https://answers.splunk.com/answers/421857/splunk-add-on-for-check-point-opsec-lea-non-audit.html

[kmuellercm]

Thank you! For some reason i didn't come across your answer in my searches.