Splunk Add-on for Check Point OPSEC LEA problem

noybin · ‎03-21-2016

Hello,

I've installed and configured the Splunk Add-on for Check Point OPSEC LEA.
I was able to pull the certificate but it never connects to the Checkpoint Firewall. In the last conection column it says "Never Connected".
I've also run a tcpdump on the splunk server and no connection is seen to the firewall. So it's not s a connectivity problem because Splunk don't even try to connect.

I've run a ./splunk cmd /sdm/splunk/etc/apps/Splunk_TA_opseclea_linux22/bin/lea-loggrabber-debug.sh and i see some error messages such as ERROR: SIC ERROR 301 - SIC Error for lea: ckpSSL ssl lib error. between others.

Please can you help me with this issue?

Thank you in advance.
Regards

Full output:
[root@tropicalia bin]# ./splunk cmd /sdm/splunk/etc/apps/Splunk_TA_opseclea_linux22/bin/lea-loggrabber-debug.sh
Using Splunk instance: /sdm/splunk, app name Splunk_TA_opseclea_linux22
Splunk username: admin
Password:
DEBUG: LOGGRABBER configuration file is: /sdm/splunk/etc/apps/Splunk_TA_opseclea_linux22/bin/fw1-loggrabber.conf
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_icmp
DEBUG: function string_duplicate
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_duplicate
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function logging_init_env
DEBUG: function open_screen
DEBUG: Open connection to screen.
DEBUG: Logfilename : fw.log
DEBUG: Record Separator : |
DEBUG: Resolve Addresses: No
DEBUG: Show Filenames : No
DEBUG: FW1-2000 : No
DEBUG: Online-Mode : Yes
DEBUG: Audit-Log : No
DEBUG: Show Fieldnames : Yes
DEBUG: function stringlist_search
DEBUG: Processing Logfile: fw.log
DEBUG: function read_fw1_logfile
splunk internal call command: $SPLUNK_HOME/bin/splunk _internal call /servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/
splunk output: QUERYING: 'https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/'
HTTP Status: 200.
Content:

<?xml version="1.0" encoding="UTF-8"?>
<!--This is to override browser formatting; see server.conf[httpServer] to disable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .-->
<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
  <title></title>
  <id>https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf</id>
  <updated>2016-03-21T13:14:50-03:00</updated>
  <generator build="f3e41e4b37b2" version="6.3.1"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/_new" rel="create"/>
  <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/_acl" rel="_acl"/>
  <opensearch:totalResults>1</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>
  <entry>
    <title>CP</title>
    <id>https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP</id>
    <updated>2016-03-21T13:14:50-03:00</updated>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="alternate"/>
    <author>
      <name>admin</name>
    </author>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="list"/>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="edit"/>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="remove"/>
    <content type="text/xml">
      <s:dict>
        <s:key name="disabled">0</s:key>
        <s:key name="eai:acl">
          <s:dict>
            <s:key name="app">Splunk_TA_opseclea_linux22</s:key>
            <s:key name="can_change_perms">1</s:key>
            <s:key name="can_list">1</s:key>
            <s:key name="can_share_app">1</s:key>
            <s:key name="can_share_global">1</s:key>
            <s:key name="can_share_user">1</s:key>
            <s:key name="can_write">1</s:key>
            <s:key name="modifiable">1</s:key>
            <s:key name="owner">admin</s:key>
            <s:key name="perms">
              <s:dict>
                <s:key name="read">
                  <s:list>
                    <s:item>admin</s:item>
                  </s:list>
                </s:key>
                <s:key name="write">
                  <s:list>
                    <s:item>admin</s:item>
                  </s:list>
                </s:key>
              </s:dict>
            </s:key>
            <s:key name="removable">1</s:key>
            <s:key name="sharing">app</s:key>
          </s:dict>
        </s:key>
        <s:key name="eai:appName">Splunk_TA_opseclea_linux22</s:key>
        <s:key name="eai:userName">nobody</s:key>
        <s:key name="fw_version">77</s:key>
        <s:key name="is_disabled">0</s:key>
        <s:key name="lea_server_auth_port">18184</s:key>
        <s:key name="lea_server_auth_type">sslca</s:key>
        <s:key name="lea_server_ip">10.10.10.201</s:key>
        <s:key name="mode">fw</s:key>
        <s:key name="no_nagle">1</s:key>
        <s:key name="online_mode">0</s:key>
        <s:key name="opsec_entity_sic_name">CN=cp_mgmt,O=pogo..4bmbx4</s:key>
        <s:key name="opsec_sic_name">CN=Splunk-Reco,O=pogo..4bmbx4</s:key>
        <s:key name="opsec_sslca_file">../certs/pogo.p12</s:key>
      </s:dict>
    </content>
  </entry>
</feed>

mode: fw
addFilter: product=VPN-1 & FireWall-1
DEBUG: function string_duplicate
-v opsec_sic_name CN=Splunk-Reco,O=pogo..4bmbx4 -v opsec_sslca_file ../certs/pogo.p12 -v lea_server ip 10.10.10.201 -v lea_server auth_port 18184 -v lea_server auth_type sslca -v lea_server opsec_entity_sic_name CN=cp_mgmt,O=pogo..4bmbx4 -v lea_server no_nagle
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Env Configuration:
(
:type (opsec_info)
:lea_server (no_nagle
:opsec_entity_sic_name ("CN=cp_mgmt,O=pogo..4bmbx4")
:auth_type (sslca)
:auth_port (18184)
:ip (10.10.10.201)
)
:opsec_sslca_file ("../certs/pogo.p12")
:opsec_sic_name ("CN=Splunk-Reco,O=pogo..4bmbx4")
)

[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Could not find info for ...opsec_shared_local_path...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Could not find info for ...opsec_sic_policy_file...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Could not find info for ...opsec_mt...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] opsec_init: multithread safety is not initialized
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprng_opsec_initialize: path is not initialized - will initialize
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprng_opsec_initialize: full file name is ops_prng
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] fwprng_opsec_read_seed: file exists but seed not initialized
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprng_opsec_initialize: dev_urandom_poll returned 0
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] opsec_file_is_intialized: seed is initialized
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprng_opsec_initialize: seed init for opsec succeeded
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_create: version 5301.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_add_name_to_group: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_set_local_names: () names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_create: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_add_name_to_group: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_set_local_names: (local_sic_name) names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_add_name_to_group: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_set_local_names: (127.0.0.1) names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_add_name_to_group: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_policy_set_local_names: ("CN=Splunk-Reco,O=pogo..4bmbx4") names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_apply_default_dn: ca_dn = [O=pogo..4bmbx4].
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_apply_default_dn: calling PM_policy_DN_conversion ..
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM_apply_default_dn: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] CkpRegDir: Environment variable CPDIR is not set.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] GenerateGlobalEntry: Unable to get registry path
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 32
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 11
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 31
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCP_Ex: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCP_Ex: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 32
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 32
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCP_Ex: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 11
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 11
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCP_Ex: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 31
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx_New: prefs = 31
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] opsec_init_sic_id_internal: Added sic id (ctx id = 0)
splunk internal call command: $SPLUNK_HOME/bin/splunk _internal call /servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/log_status/1@
splunk output: QUERYING: 'https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/log_status/1@'
FAILED: 'HTTP/1.1 404 Not Found'
Content:

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">
 In handler 'log_status': Could not find object id=1@</msg>
  </messages>
</response>

splunkd request failed, 404:
$SPLUNK_HOME/bin/splunk _internal call /servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/log_status/1@
QUERYING: 'https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/log_status/1@'
FAILED: 'HTTP/1.1 404 Not Found'
Content:

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">
 In handler 'log_status': Could not find object id=1@</msg>
  </messages>
</response>

DEBUG: Starting fw.log 1 at offset -1
DEBUG: OPSEC LEA conf file is lea.conf
DEBUG: Authentication mode has been used.
DEBUG: Server-IP : 10.10.10.201
DEBUG: Server-Port : 18184
DEBUG: Authentication type: sslca
DEBUG: OPSEC sic certificate file name : ../certs/pogo.p12
DEBUG: Server DN (sic name) : CN=cp_mgmt,O=pogo..4bmbx4
DEBUG: OPSEC LEA client DN (sic name) : CN=Splunk-Reco,O=pogo..4bmbx4
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_init_entity_sic: called for the client side
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Configuring entity lea_server
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...conn_buf_size...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...no_nagle...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...port...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_entity_add_sic_rule: adding rules: apply_to: ME, peer: CN=cp_mgmt,O=pogo..4bmbx4, d_ip: NULL, dport 18184, svc: lea, method: sslca
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_entity_add_sic_rule: adding INBOUND rule
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_entity_add_sic_rule: adding OUTBOUND rule
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_get_comm: creating comm for ent=9ff18b8 peer=9ffc8a8 passive=0 key=2 info=0
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] c=0x9ff18b8 s=0x9ffc8a8 comm_type=4

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...opsec_client...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_get_comm: Creating session hash (size=256)
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_get_comm: ADDING comm=0x9fe7e40 to ent=0x9ff18b8 with key=2
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_env_get_context_id_by_peer_sic_name: found context id=0 for peer sic name=CN=cp_mgmt,O=pogo..4bmbx4
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_env_get_sic_handle_by_context_id: found sic handle (ctx id=0)
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec_sic_connect: connecting... (ctx id=0)
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] resolver_gethostbyname: Performing gethostbyname for tropicalia
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] peers addresses are
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] 192.168.4.100
DEBUG: function read_fw1_logfile_start
DEBUG: OPSEC session start handler was invoked
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] SESSION ID:3 is sending DG_TYPE=1

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] pushing dgtype=1 len=0 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] SESSION ID:3 is sending DG_TYPE=402

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] pushing dgtype=402 len=27 to list=0x9fe7e5c
filter 0: product=VPN-1 & FireWall-1
DEBUG: function create_fw1_filter_rule
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
DEBUG: function string_get_token
DEBUG: function string_trim
DEBUG: function string_left_trim
DEBUG: function string_right_trim
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] SESSION ID:3 is sending DG_TYPE=40f

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] pushing dgtype=40f len=139 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] fwasync_conn_params: ->
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] fwasync_connbuf_realloc: reallocating 0 from 0 to 1028
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] fwasync_connbuf_realloc: reallocating 0 from 0 to 1028
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] sic_client_set_version: 10: protocol version is 59000000
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] call_handlers_list: no conversion done, set CN=cp_mgmt,O=pogo..4bmbx4 as sic name
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM_session_init: given session O(CN=Splunk-Reco,O=pogo..4bmbx4;CN=cp_mgmt,O=pogo..4bmbx4;18184;lea).
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM_policy_query: input session O(CN=Splunk-Reco,O=pogo..4bmbx4;CN=cp_mgmt,O=pogo..4bmbx4;18184;lea).
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM_policy_query: rule found (ME;CN=cp_mgmt,O=pogo..4bmbx4;18184;lea;sslca(1/1)).
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM_policy_query: finished successfully. 1st method = sslca
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM_policy_choose: finished successfully. choose: sslca.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] do_getver: can't get inode of .//session.NDB: No such file or directory
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] sslca_read_session: failed to get cached session
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] auth_sslca_client_handler: failed to read session
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] ckpSSL_PrepareConnection: verify mode: 3
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] My SSL Ciphers:
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] Cipher List:
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] 0: DES-CBC3-SHA SSLv3 Kx=RSA Au=RSA Enc=3DES(168) Mac=SHA1

[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] 1: RC4-SHA SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=SHA1

[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] 2: RC4-MD5 SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5

[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] ckpSSL_NegotiateStep: current state = before/connect initialization
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] is_initialized: new process or forked
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] fwprng_get_entropy_collection_time_opsec: value read is 0
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] cpprng_get_opsec_entropy_collection_time: entropy_collection time returned is 0
[ 9348 4149401280]@tropicalia[21 Mar 13:15:40] fwprng_set_entropy_collection_time_opsec: entering time is Mon Mar 21 13:15:40 2016 (1458576940)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSL_fwasync_connected: no connections err -3
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSL_fwasync_close: start shutdown
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] sic_client_end_handler: for conn id = 10
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected: connect failed (301)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected: SIC Error for lea: ckpSSL ssl lib error
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected:conn=(nil) opaque=0x9ffc838 err=0 comm=0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] comm failed to connect 0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] OPSEC_SET_ERRNO: err = 8 Comm is not connected/Unable to connect (pre = 0)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] COM 0x9fe7e40 got signal 131075
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] destroying comm 0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying comm 0x9fe7e40 with 1 active sessions
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying session (a0000c8) id 3 (ent=9ff18b8) reason=SIC_FAILURE
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] SESSION ID:3 is sending DG_TYPE=3

DEBUG: OPSEC_SESSION_END_HANDLER called
ERROR: SIC ERROR 301 - SIC Error for lea: ckpSSL ssl lib error
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_comm_is_needed:comm 0x9fe7e40 1/1 sessions need the comm.
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=1 len=0 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=402 len=27 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=40f len=139 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=ffffffff len=-1 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] REMOVING comm=0x9fe7e40 from ent=0x9ff18b8 with key=2
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSL_ShutdownHandler: rc=1 (0) SSLv3 read server hello A
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSL_ShutdownHandler: sync shutdown (fd=10)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSL_Destroy: closed fd 10
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] T_event_mainloop_e: T_event_mainloop_iter returns 0
DEBUG: function cleanup_fw1_environment
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying entity 1 with 0 active comms
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_destroy_entity_sic: deleting sic rules for entity 0x9ff18b8
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying entity 2 with 0 active comms
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_destroy_entity_sic: deleting sic rules for entity 0x9ffc8a8
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] IpcUnMapFile: unmapping file (handle=0x9fe7768)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe7848)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe78c8)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe7968)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe7c90)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] PM_policy_destroy: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] opsec_destroy_sic_id_internal: Destroyed sic id (ctx id=0)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] opsec_env_destroy_sic_id_hash: Destroyed sic id hash
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] fwd_env_destroy: env 0x9fcb108 (alloced = 1)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] T_env_destroy: env 0x9fcb108
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] do_fwd_env_destroy: really destroy 0x9fcb108
DEBUG: function close_screen
DEBUG: Close connection to screen.
DEBUG: function exit_loggrabber
DEBUG: function free_lfield_arrays
DEBUG: function free_afield_arrays
DEBUG: function free_lfield_arrays

DEBUG: function free_afield_arrays

This is the opsec.conf:
[root@tropicalia ~]# cat /sdm/splunk/etc/apps/Splunk_TA_opseclea_linux22/local/opsec.conf
[CP]
fw_version = 77
is_disabled = 0
lea_server_auth_port = 18184
lea_server_auth_type = sslca
lea_server_ip = 10.10.10.201
mode = fw
online_mode = 0
opsec_entity_sic_name = CN=cp_mgmt,O=pogo..4bmbx4
opsec_sic_name = CN=Splunk-Reco,O=pogo..4bmbx4
opsec_sslca_file = ../certs/pogo.p12
disabled = 0

no_nagle = 1

I've attached the connection configuration:

ryandg · ‎04-14-2016

Based on your logs it is trying to connect to the Opsec server but the connection gets refused. If you're able to pull the cert then my current best guess is that the Entity SIC Name is wrong -- some CP Admins make the FWs have custom names. For example, they might've named it TropicalLA or something also this is case sensitive keep in mind.

noybin · ‎04-14-2016

Hi ryandg,

I don't think that it is a connectivity problem because when I run a tcpdump on the splunk server, I don't see any attempt to connect to the firewall. So splunk is not trying to reach the firewall at all.

Thank you.

ryandg · ‎04-14-2016

If you restart splunkd while running a tcp dump, you see zero packets reaching out to the server? It just seems strange because according to your logs:

[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected: connect failed (301)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected: SIC Error for lea: ckpSSL ssl lib error
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected:conn=(nil) opaque=0x9ffc838 err=0 comm=0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] comm failed to connect 0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] OPSEC_SET_ERRNO: err = 8 Comm is not connected/Unable to connect (pre = 0)

It thinks it is connecting out and attempting to reach them. Can you try starting a tcpdump on one session, double check the port and dump query and then in a second session run a splunkd restart?

jpvlsmv · ‎04-14-2016

Could iptables (or other host-based firewall) or apparmor or SE policies be preventing the splunk service (and specifically the lea_log_grabber.sh that runs under it) be blocking outbound connections?

noybin · ‎04-14-2016

I've just done that and nothing is seen on the tcpdump output.

tcpdump -vi ens32 host 10.10.10.201

ryandg · ‎04-14-2016

Do you have any other CMAs/CLMs?

noybin · ‎04-14-2016

I am asking to the FW admin. I will write as soon as he answers me.

I also want to add that we are running Splunk on a Centos 7 and we followed the procedure below when installing the app:

https://answers.splunk.com/answers/89697/check-point-ospec-lea-app-bad-elf-interpreter-error.html

noybin · ‎04-14-2016

I've ran the following:

/etc/apps/Splunk_TA_opseclea_linux22/bin/lea-loggrabber-debug.sh

And that returned a lot of data. So I think that it is not a connectivity problem. The following is a little part of the output:

loc=391468|time=14Apr2016 15:57:43|action=accept|orig=salsa|i/f_dir=inbound|i/f_name=eth3.101|has_accounting=0|product=VPN-1 & FireWall-1|inzone=Internal|outzone=External|rule=9|rule_uid={1B559F21-9B45-4568-AB00-632D730B4B95}|session_id:=3191|dns_query=wildcard.adroll.com.edgekey.net |dns_type=A|service_id=domain-udp|src=guajira|s_port=36636|dst=208.67.220.220|service=domain-udp|proto=udp|xlatesrc=IP_Telmex_201|xlatesport=Unknown|xlatedport=Unknown|NAT_rulenum=29|NAT_addtnl_rulenum=1|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={C40D4BFA-4622-7247-ABD7-9B14BC334ED2};mgmt=pogo;date=1460468083;policy_name=R77-AR]|origin_sic_name=CN=salsa,O=pogo..4bmbx4
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_do_read: read 12 bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_do_read: read 455 bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] demultiplex type=505 session-id=3
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] client: got RECORD session 3
DEBUG: function read_fw1_logfile_record
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function submit_screen
DEBUG: Submit message to screen.
loc=391469|time=14Apr2016 15:57:43|action=drop|orig=salsa|i/f_dir=inbound|i/f_name=eth2.106|has_accounting=0|product=VPN-1 & FireWall-1|rule=243|rule_uid={460EDE04-17FF-49CA-A722-360A0D25294D}|src=Video-SRV|s_port=nbdatagram|dst=192.168.6.255|service=nbdatagram|proto=udp|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={C40D4BFA-4622-7247-ABD7-9B14BC334ED2};mgmt=pogo;date=1460468083;policy_name=R77-AR]|origin_sic_name=CN=salsa,O=pogo..4bmbx4

noybin · ‎04-14-2016

I solved the problem.

Solution:
1.Set the environment variable $SPLUNK_HOME
2. Create a new connection
3. Pull the certificate again.

Thanks for your help.