All Apps and Add-ons

Splunk Add-on for Check Point OPSEC LEA 4.0.0: "UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 330: invalid continuation byte"

kmuellercm
Explorer

R77 with dedicated logging server
Enabled a LEA connection and I get just a few logs, then the process bombs out. Logs from var/log/splunk/splunk_ta_checkpoint-opseclea_modinput.log indicate EventWriter encountered an exception, then stops processing inputs. I get about 150k log lines parsed properly. No obvious errors in splunkd.log and the loggrabber process remains running:

 /opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/../bin/lea_loggrabber --data non_audit --debug_level 2 --appname Splunk_TA_checkpoint-opseclea --lea_server_ip 1.2.3.4 --lea_server_auth_port 18184 --lea_server_auth_type sslca --opsec_sslca_file /opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/certs/SplunkLEA.p12 --opsec_sic_name CN=SplunkLEA--opsec_entity_sic_name CN=MGMTSVR --last_record_location 1471966084 115426 --online --no_resolve

var/log/splunk/splunk_ta_checkpoint-opseclea_modinput.log:

2016-08-23 16:13:16,654 +0000 log_level=INFO, pid=17396, tid=Thread-9, file=ta_opseclea_data_collector.py, func_name=get_logs, code_line_no=62 | [input_name="Checkpoint NonAudit Events" connection="SplunkLEA-dedicated" data="non_audit"]log_level=2 file:lea_loggrabber.cpp func_name:read_fw1_logfile_collogs code_line_no:2052 :LEA collected logfile handler was invoked
2016-08-23 16:13:27,679 +0000 log_level=ERROR, pid=17396, tid=Thread-1, file=event_writer.py, func_name=_do_write_events, code_line_no=79 | EventWriter encounter exception which maycause data loss, queue leftsize=3
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktalib/event_writer.py", line 62, in _do_write_events
    for evt in event:
  File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktaucclib/data_collection/ta_data_collector.py", line 59, in <genexpr>
    index, scu.escape_cdata(event.event)) for event
  File "/opt/splunk/etc/apps/Splunk_TA_checkpoint-opseclea/bin/splunk_ta_checkpoint_opseclea/splunktalib/common/util.py", line 71, in escape_cdata
    data = data.encode("utf-8", errors="xmlcharrefreplace")
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 330: invalid continuation byte
2016-08-23 16:13:27,679 +0000 log_level=INFO, pid=17396, tid=Thread-1, file=event_writer.py, func_name=_do_write_events, code_line_no=84 | Event writer stopped, queue leftsize=4
2016-08-23 16:13:27,680 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=ta_data_collector.py, func_name=_write_events, code_line_no=122 | [input_name="Checkpoint NonAudit Events" data="non_audit"]  the event queue is closed and the received data will be discarded
2016-08-23 16:13:27,681 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=ta_data_collector.py, func_name=index_data, code_line_no=114 | [input_name="Checkpoint NonAudit Events" data="non_audit"]  End of indexing data for Checkpoint NonAudit Events_non_audit
2016-08-23 16:13:27,681 +0000 log_level=INFO, pid=17396, tid=Thread-4, file=thread_pool.py, func_name=_run, code_line_no=261 | Thread work_queue_size=0
0 Karma
1 Solution

jamesarmitage
Path Finder

It looks like you're encountering the same issue I did. I made a workaround that requires modifying a single line in the TA:

https://answers.splunk.com/answers/421857/splunk-add-on-for-check-point-opsec-lea-non-audit.html

View solution in original post

0 Karma

jamesarmitage
Path Finder

It looks like you're encountering the same issue I did. I made a workaround that requires modifying a single line in the TA:

https://answers.splunk.com/answers/421857/splunk-add-on-for-check-point-opsec-lea-non-audit.html

0 Karma

kmuellercm
Explorer

Thank you! For some reason i didn't come across your answer in my searches.

0 Karma
Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...