Archive

Splunk Add-on for Check Point OPSEC LEA problem

Communicator

Hello,

I've installed and configured the Splunk Add-on for Check Point OPSEC LEA.
I was able to pull the certificate but it never connects to the Checkpoint Firewall. In the last conection column it says "Never Connected".
I've also run a tcpdump on the splunk server and no connection is seen to the firewall. So it's not s a connectivity problem because Splunk don't even try to connect.

I've run a ./splunk cmd /sdm/splunk/etc/apps/SplunkTAopseclea_linux22/bin/lea-loggrabber-debug.sh and i see some error messages such as ERROR: SIC ERROR 301 - SIC Error for lea: ckpSSL ssl lib error. between others.

Please can you help me with this issue?

Thank you in advance.
Regards

Full output:
[root@tropicalia bin]# ./splunk cmd /sdm/splunk/etc/apps/SplunkTAopseclealinux22/bin/lea-loggrabber-debug.sh
Using Splunk instance: /sdm/splunk, app name Splunk
TAopseclealinux22
Splunk username: admin
Password:
DEBUG: LOGGRABBER configuration file is: /sdm/splunk/etc/apps/SplunkTAopseclealinux22/bin/fw1-loggrabber.conf
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
duplicate
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
duplicate
DEBUG: function stringicmp
DEBUG: function string
duplicate
DEBUG: function stringduplicate
DEBUG: function string
icmp
DEBUG: function stringduplicate
DEBUG: function string
duplicate
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
duplicate
DEBUG: function stringicmp
DEBUG: function string
duplicate
DEBUG: function stringduplicate
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringduplicate
DEBUG: function string
icmp
DEBUG: function stringduplicate
DEBUG: function string
duplicate
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
duplicate
DEBUG: function stringicmp
DEBUG: function string
duplicate
DEBUG: function stringduplicate
DEBUG: function string
icmp
DEBUG: function stringduplicate
DEBUG: function string
duplicate
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
duplicate
DEBUG: function stringicmp
DEBUG: function string
duplicate
DEBUG: function stringduplicate
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
duplicate
DEBUG: function stringicmp
DEBUG: function string
duplicate
DEBUG: function stringduplicate
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
duplicate
DEBUG: function stringicmp
DEBUG: function string
duplicate
DEBUG: function stringduplicate
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringduplicate
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function logging
initenv
DEBUG: function open
screen
DEBUG: Open connection to screen.
DEBUG: Logfilename : fw.log
DEBUG: Record Separator : |
DEBUG: Resolve Addresses: No
DEBUG: Show Filenames : No
DEBUG: FW1-2000 : No
DEBUG: Online-Mode : Yes
DEBUG: Audit-Log : No
DEBUG: Show Fieldnames : Yes
DEBUG: function stringlistsearch
DEBUG: Processing Logfile: fw.log
DEBUG: function read
fw1logfile
splunk internal call command: $SPLUNK
HOME/bin/splunk internal call /servicesNS/nobody/SplunkTAopseclealinux22/opsec/opsecconf/
splunk output: QUERYING: 'https://127.0.0.1:8089/servicesNS/nobody/Splunk
TAopseclealinux22/opsec/opsec_conf/'
HTTP Status: 200.
Content:

<?xml version="1.0" encoding="UTF-8"?>
<!--This is to override browser formatting; see server.conf[httpServer] to disable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .-->
<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
  <title></title>
  <id>https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf</id>
  <updated>2016-03-21T13:14:50-03:00</updated>
  <generator build="f3e41e4b37b2" version="6.3.1"/>
  <author>
    <name>Splunk</name>
  </author>
  <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/_new" rel="create"/>
  <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/_acl" rel="_acl"/>
  <opensearch:totalResults>1</opensearch:totalResults>
  <opensearch:itemsPerPage>30</opensearch:itemsPerPage>
  <opensearch:startIndex>0</opensearch:startIndex>
  <s:messages/>
  <entry>
    <title>CP</title>
    <id>https://127.0.0.1:8089/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP</id>
    <updated>2016-03-21T13:14:50-03:00</updated>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="alternate"/>
    <author>
      <name>admin</name>
    </author>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="list"/>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="edit"/>
    <link href="/servicesNS/nobody/Splunk_TA_opseclea_linux22/opsec/opsec_conf/CP" rel="remove"/>
    <content type="text/xml">
      <s:dict>
        <s:key name="disabled">0</s:key>
        <s:key name="eai:acl">
          <s:dict>
            <s:key name="app">Splunk_TA_opseclea_linux22</s:key>
            <s:key name="can_change_perms">1</s:key>
            <s:key name="can_list">1</s:key>
            <s:key name="can_share_app">1</s:key>
            <s:key name="can_share_global">1</s:key>
            <s:key name="can_share_user">1</s:key>
            <s:key name="can_write">1</s:key>
            <s:key name="modifiable">1</s:key>
            <s:key name="owner">admin</s:key>
            <s:key name="perms">
              <s:dict>
                <s:key name="read">
                  <s:list>
                    <s:item>admin</s:item>
                  </s:list>
                </s:key>
                <s:key name="write">
                  <s:list>
                    <s:item>admin</s:item>
                  </s:list>
                </s:key>
              </s:dict>
            </s:key>
            <s:key name="removable">1</s:key>
            <s:key name="sharing">app</s:key>
          </s:dict>
        </s:key>
        <s:key name="eai:appName">Splunk_TA_opseclea_linux22</s:key>
        <s:key name="eai:userName">nobody</s:key>
        <s:key name="fw_version">77</s:key>
        <s:key name="is_disabled">0</s:key>
        <s:key name="lea_server_auth_port">18184</s:key>
        <s:key name="lea_server_auth_type">sslca</s:key>
        <s:key name="lea_server_ip">10.10.10.201</s:key>
        <s:key name="mode">fw</s:key>
        <s:key name="no_nagle">1</s:key>
        <s:key name="online_mode">0</s:key>
        <s:key name="opsec_entity_sic_name">CN=cp_mgmt,O=pogo..4bmbx4</s:key>
        <s:key name="opsec_sic_name">CN=Splunk-Reco,O=pogo..4bmbx4</s:key>
        <s:key name="opsec_sslca_file">../certs/pogo.p12</s:key>
      </s:dict>
    </content>
  </entry>
</feed>

mode: fw
addFilter: product=VPN-1 & FireWall-1
DEBUG: function stringduplicate
-v opsec
sicname CN=Splunk-Reco,O=pogo..4bmbx4 -v opsecsslcafile ../certs/pogo.p12 -v leaserver ip 10.10.10.201 -v leaserver authport 18184 -v leaserver authtype sslca -v leaserver opsecentitysicname CN=cpmgmt,O=pogo..4bmbx4 -v leaserver nonagle
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Env Configuration:
(
:type (opsec
info)
:leaserver (nonagle
:opsecentitysicname ("CN=cpmgmt,O=pogo..4bmbx4")
:authtype (sslca)
:auth
port (18184)
:ip (10.10.10.201)
)
:opsecsslcafile ("../certs/pogo.p12")
:opsecsicname ("CN=Splunk-Reco,O=pogo..4bmbx4")
)

[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Could not find info for ...opsecsharedlocalpath...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Could not find info for ...opsec
sicpolicyfile...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] Could not find info for ...opsecmt...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] opsec
init: multithread safety is not initialized
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprngopsecinitialize: path is not initialized - will initialize
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprngopsecinitialize: full file name is opsprng
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] fwprng
opsecreadseed: file exists but seed not initialized
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprngopsecinitialize: devurandompoll returned 0
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] opsecfileisintialized: seed is initialized
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] cpprng
opsecinitialize: seed init for opsec succeeded
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
policycreate: version 5301.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
policyaddnametogroup: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PMpolicysetlocalnames: () names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PMpolicycreate: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PMpolicyaddnametogroup: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
policysetlocalnames: (localsicname) names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
policyaddnametogroup: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PMpolicysetlocalnames: (127.0.0.1) names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PMpolicyaddnametogroup: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
policysetlocalnames: ("CN=Splunk-Reco,O=pogo..4bmbx4") names. finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
applydefaultdn: cadn = [O=pogo..4bmbx4].
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
applydefaultdn: calling PMpolicyDNconversion ..
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] PM
applydefaultdn: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctxNew: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] CkpRegDir: Environment variable CPDIR is not set.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] GenerateGlobalEntry: Unable to get registry path
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx
New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctxNew: prefs = 32
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx
New: prefs = 11
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctxNew: prefs = 31
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx
New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCPEx: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx
New: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctxNew: prefs = 12
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCP
Ex: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctxNew: prefs = 32
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx
New: prefs = 32
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCPEx: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx
New: prefs = 11
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctxNew: prefs = 11
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] sslcaInitCP
Ex: using asym client without ca cert
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctxNew: prefs = 31
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] ckpSSLctx
New: prefs = 31
[ 9348 4149401280]@tropicalia[21 Mar 13:14:50] opsecinitsicidinternal: Added sic id (ctx id = 0)
splunk internal call command: $SPLUNKHOME/bin/splunk _internal call /servicesNS/nobody/SplunkTAopseclealinux22/opsec/logstatus/1@
splunk output: QUERYING: 'https://127.0.0.1:8089/servicesNS/nobody/Splunk
TAopseclealinux22/opsec/log_status/1@'
FAILED: 'HTTP/1.1 404 Not Found'
Content:

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">
 In handler 'log_status': Could not find object id=1@</msg>
  </messages>
</response>

splunkd request failed, 404:
$SPLUNKHOME/bin/splunk _internal call /servicesNS/nobody/SplunkTAopseclealinux22/opsec/logstatus/1@
QUERYING: 'https://127.0.0.1:8089/servicesNS/nobody/Splunk
TAopseclealinux22/opsec/log_status/1@'
FAILED: 'HTTP/1.1 404 Not Found'
Content:

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">
 In handler 'log_status': Could not find object id=1@</msg>
  </messages>
</response>

DEBUG: Starting fw.log 1 at offset -1
DEBUG: OPSEC LEA conf file is lea.conf
DEBUG: Authentication mode has been used.
DEBUG: Server-IP : 10.10.10.201
DEBUG: Server-Port : 18184
DEBUG: Authentication type: sslca
DEBUG: OPSEC sic certificate file name : ../certs/pogo.p12
DEBUG: Server DN (sic name) : CN=cpmgmt,O=pogo..4bmbx4
DEBUG: OPSEC LEA client DN (sic name) : CN=Splunk-Reco,O=pogo..4bmbx4
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
initentitysic: called for the client side
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Configuring entity leaserver
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...conn
bufsize...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...no
nagle...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...port...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsecentityaddsicrule: adding rules: applyto: ME, peer: CN=cpmgmt,O=pogo..4bmbx4, dip: NULL, dport 18184, svc: lea, method: sslca
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
entityaddsicrule: adding INBOUND rule
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
entityaddsicrule: adding OUTBOUND rule
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
getcomm: creating comm for ent=9ff18b8 peer=9ffc8a8 passive=0 key=2 info=0
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] c=0x9ff18b8 s=0x9ffc8a8 comm
type=4

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] Could not find info for ...opsecclient...
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
getcomm: Creating session hash (size=256)
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
getcomm: ADDING comm=0x9fe7e40 to ent=0x9ff18b8 with key=2
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
envgetcontextidbypeersicname: found context id=0 for peer sic name=CN=cpmgmt,O=pogo..4bmbx4
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsecenvgetsichandlebycontextid: found sic handle (ctx id=0)
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] opsec
sicconnect: connecting... (ctx id=0)
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] resolver
gethostbyname: Performing gethostbyname for tropicalia
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] peers addresses are
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] 192.168.4.100
DEBUG: function readfw1logfilestart
DEBUG: OPSEC session start handler was invoked
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] SESSION ID:3 is sending DG
TYPE=1

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] pushing dgtype=1 len=0 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] SESSION ID:3 is sending DG_TYPE=402

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] pushing dgtype=402 len=27 to list=0x9fe7e5c
filter 0: product=VPN-1 & FireWall-1
DEBUG: function createfw1filterrule
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
DEBUG: function stringtrim
DEBUG: function string
lefttrim
DEBUG: function string
righttrim
DEBUG: function string
gettoken
DEBUG: function string
trim
DEBUG: function stringlefttrim
DEBUG: function stringrighttrim
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] SESSION ID:3 is sending DG_TYPE=40f

[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] pushing dgtype=40f len=139 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] fwasyncconnparams: ->
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] fwasyncconnbufrealloc: reallocating 0 from 0 to 1028
[ 9348 4149401280]@tropicalia[21 Mar 13:14:52] fwasyncconnbufrealloc: reallocating 0 from 0 to 1028
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] sicclientsetversion: 10: protocol version is 59000000
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] call
handlerslist: no conversion done, set CN=cpmgmt,O=pogo..4bmbx4 as sic name
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PMsessioninit: given session O(CN=Splunk-Reco,O=pogo..4bmbx4;CN=cpmgmt,O=pogo..4bmbx4;18184;lea).
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM
policyquery: input session O(CN=Splunk-Reco,O=pogo..4bmbx4;CN=cpmgmt,O=pogo..4bmbx4;18184;lea).
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PMpolicyquery: rule found (ME;CN=cpmgmt,O=pogo..4bmbx4;18184;lea;sslca(1/1)).
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM
policyquery: finished successfully. 1st method = sslca
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] PM
policychoose: finished successfully. choose: sslca.
[ 9348 4149401280]@tropicalia[21 Mar 13:14:53] do
getver: can't get inode of .//session.NDB: No such file or directory
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] sslcareadsession: failed to get cached session
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] authsslcaclienthandler: failed to read session
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] ckpSSL
PrepareConnection: verify mode: 3
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] My SSL Ciphers:
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] Cipher List:
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] 0: DES-CBC3-SHA SSLv3 Kx=RSA Au=RSA Enc=3DES(168) Mac=SHA1

[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] 1: RC4-SHA SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=SHA1

[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] 2: RC4-MD5 SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5

[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] ckpSSLNegotiateStep: current state = before/connect initialization
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] is
initialized: new process or forked
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] fwprnggetentropycollectiontimeopsec: value read is 0
[ 9348 4149401280]@tropicalia[21 Mar 13:15:24] cpprng
getopsecentropycollectiontime: entropycollection time returned is 0
[ 9348 4149401280]@tropicalia[21 Mar 13:15:40] fwprng
setentropycollectiontimeopsec: entering time is Mon Mar 21 13:15:40 2016 (1458576940)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSLfwasyncconnected: no connections err -3
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSLfwasyncclose: start shutdown
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] sicclientendhandler: for conn id = 10
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec
authclientconnected: connect failed (301)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsecauthclientconnected: SIC Error for lea: ckpSSL ssl lib error
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec
authclientconnected:conn=(nil) opaque=0x9ffc838 err=0 comm=0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] comm failed to connect 0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] OPSECSETERRNO: err = 8 Comm is not connected/Unable to connect (pre = 0)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] COM 0x9fe7e40 got signal 131075
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] destroying comm 0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying comm 0x9fe7e40 with 1 active sessions
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying session (a0000c8) id 3 (ent=9ff18b8) reason=SICFAILURE
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] SESSION ID:3 is sending DG
TYPE=3

DEBUG: OPSECSESSIONENDHANDLER called
ERROR: SIC ERROR 301 - SIC Error for lea: ckpSSL ssl lib error
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec
commisneeded:comm 0x9fe7e40 1/1 sessions need the comm.
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=1 len=0 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=402 len=27 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=40f len=139 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] pulling dgtype=ffffffff len=-1 to list=0x9fe7e5c
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] REMOVING comm=0x9fe7e40 from ent=0x9ff18b8 with key=2
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSLShutdownHandler: rc=1 (0) SSLv3 read server hello A
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSL
ShutdownHandler: sync shutdown (fd=10)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] ckpSSLDestroy: closed fd 10
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] T
eventmainloope: Teventmainloopiter returns 0
DEBUG: function cleanup
fw1environment
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying entity 1 with 0 active comms
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec
destroyentitysic: deleting sic rules for entity 0x9ff18b8
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] Destroying entity 2 with 0 active comms
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsecdestroyentitysic: deleting sic rules for entity 0x9ffc8a8
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] IpcUnMapFile: unmapping file (handle=0x9fe7768)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe7848)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe78c8)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe7968)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] IpcUnMapFile: unmapping file (handle=0x9fe7c90)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] PM
policydestroy: finished successfully.
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] opsec
destroysicidinternal: Destroyed sic id (ctx id=0)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] opsec
envdestroysicidhash: Destroyed sic id hash
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] fwdenvdestroy: env 0x9fcb108 (alloced = 1)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] Tenvdestroy: env 0x9fcb108
[ 9348 4149401280]@tropicalia[21 Mar 13:15:53] dofwdenvdestroy: really destroy 0x9fcb108
DEBUG: function close
screen
DEBUG: Close connection to screen.
DEBUG: function exitloggrabber
DEBUG: function free
lfieldarrays
DEBUG: function free
afieldarrays
DEBUG: function free
lfield_arrays

DEBUG: function freeafieldarrays

This is the opsec.conf:
[root@tropicalia ~]# cat /sdm/splunk/etc/apps/SplunkTAopseclealinux22/local/opsec.conf
[CP]
fw
version = 77
isdisabled = 0
lea
serverauthport = 18184
leaserverauthtype = sslca
lea
serverip = 10.10.10.201
mode = fw
online
mode = 0
opsecentitysicname = CN=cpmgmt,O=pogo..4bmbx4
opsecsicname = CN=Splunk-Reco,O=pogo..4bmbx4
opsecsslcafile = ../certs/pogo.p12
disabled = 0

no_nagle = 1

I've attached the connection configuration:
alt text

Communicator

Based on your logs it is trying to connect to the Opsec server but the connection gets refused. If you're able to pull the cert then my current best guess is that the Entity SIC Name is wrong -- some CP Admins make the FWs have custom names. For example, they might've named it TropicalLA or something also this is case sensitive keep in mind.

0 Karma

Communicator

Hi ryandg,

I don't think that it is a connectivity problem because when I run a tcpdump on the splunk server, I don't see any attempt to connect to the firewall. So splunk is not trying to reach the firewall at all.

Thank you.

0 Karma

Communicator

If you restart splunkd while running a tcp dump, you see zero packets reaching out to the server? It just seems strange because according to your logs:

[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected: connect failed (301)
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected: SIC Error for lea: ckpSSL ssl lib error
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] opsec_auth_client_connected:conn=(nil) opaque=0x9ffc838 err=0 comm=0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] comm failed to connect 0x9fe7e40
[ 9348 4149401280]@tropicalia[21 Mar 13:15:48] OPSEC_SET_ERRNO: err = 8 Comm is not connected/Unable to connect (pre = 0)

It thinks it is connecting out and attempting to reach them. Can you try starting a tcpdump on one session, double check the port and dump query and then in a second session run a splunkd restart?

0 Karma

Path Finder

Could iptables (or other host-based firewall) or apparmor or SE policies be preventing the splunk service (and specifically the lealoggrabber.sh that runs under it) be blocking outbound connections?

0 Karma

Communicator

I've just done that and nothing is seen on the tcpdump output.

tcpdump -vi ens32 host 10.10.10.201

0 Karma

Communicator

Do you have any other CMAs/CLMs?

0 Karma

Communicator

I am asking to the FW admin. I will write as soon as he answers me.

I also want to add that we are running Splunk on a Centos 7 and we followed the procedure below when installing the app:

https://answers.splunk.com/answers/89697/check-point-ospec-lea-app-bad-elf-interpreter-error.html

0 Karma

Communicator

I've ran the following:

/etc/apps/Splunk_TA_opseclea_linux22/bin/lea-loggrabber-debug.sh

And that returned a lot of data. So I think that it is not a connectivity problem. The following is a little part of the output:

loc=391468|time=14Apr2016 15:57:43|action=accept|orig=salsa|i/f_dir=inbound|i/f_name=eth3.101|has_accounting=0|product=VPN-1 & FireWall-1|inzone=Internal|outzone=External|rule=9|rule_uid={1B559F21-9B45-4568-AB00-632D730B4B95}|session_id:=3191|dns_query=wildcard.adroll.com.edgekey.net |dns_type=A|service_id=domain-udp|src=guajira|s_port=36636|dst=208.67.220.220|service=domain-udp|proto=udp|xlatesrc=IP_Telmex_201|xlatesport=Unknown|xlatedport=Unknown|NAT_rulenum=29|NAT_addtnl_rulenum=1|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={C40D4BFA-4622-7247-ABD7-9B14BC334ED2};mgmt=pogo;date=1460468083;policy_name=R77-AR]|origin_sic_name=CN=salsa,O=pogo..4bmbx4
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_do_read: read 12 bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_InputPending 1 pending bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] ckpSSL_do_read: read 455 bytes
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] demultiplex type=505 session-id=3
[ 21653 4149450432]@tropicalia[14 Apr 15:57:45] client: got RECORD session 3
DEBUG: function read_fw1_logfile_record
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function string_escape
DEBUG: function string_escape
DEBUG: function string_cat
DEBUG: function submit_screen
DEBUG: Submit message to screen.
loc=391469|time=14Apr2016 15:57:43|action=drop|orig=salsa|i/f_dir=inbound|i/f_name=eth2.106|has_accounting=0|product=VPN-1 & FireWall-1|rule=243|rule_uid={460EDE04-17FF-49CA-A722-360A0D25294D}|src=Video-SRV|s_port=nbdatagram|dst=192.168.6.255|service=nbdatagram|proto=udp|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={C40D4BFA-4622-7247-ABD7-9B14BC334ED2};mgmt=pogo;date=1460468083;policy_name=R77-AR]|origin_sic_name=CN=salsa,O=pogo..4bmbx4
0 Karma

Communicator

I solved the problem.

Solution:
1.Set the environment variable $SPLUNK_HOME
2. Create a new connection
3. Pull the certificate again.

Thanks for your help.

0 Karma