Getting Data In

Cisco IPS Error [errno="" 8]

seanp
Path Finder

I have been attempting to setup the Cisco IPS app for Splunk 6. However I am getting the following error in the sdee_get.log:

INFO - Checking for exsisting SubscriptionID on host: <IPADDRESS>
INFO - No exsisting SubscriptionID for host: <IPADDRESS>
INFO - Attempting to connect to sensor: <IPADDRESS>
INFO - Successfully connected to: <IPADDRESS>
ERROR - Connecting to sensor - <IPADDRESS>: URLError: <urlopen error [Errno 8] _ssl.c:521: EOF occurred in violation of protocol>

where is the IP address of the IPS. Does anyone have any thoughts into what the error is? Any help is greatly appreciated

Tags (2)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

This looks a whole lot like https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371 which seems to be a bug in OpenSSL when attempting to do TLS version renegotiation. The bug was fixed in OpenSSL upstream and in Debian / Ubuntu.

But, Splunk ships with its own version of OpenSSL. In Splunk 6.0.0 it seems to be OpenSSL 1.0.1e, which is likely affected by this issue.

Ther launchpad link above suggests some (very very very hackish) workarounds like updating python standard library files. I would personally open a support case w/ Splunk and in the meanwhile perhaps downgrade to Splunk 5.0.5, which has an older OpenSSL. Or, you could install a 5.0.5 forwarder just for your IPS app...

View solution in original post

pbalsley
Path Finder

I had the same error but only on some of my IPS. I noticed the ones that worked are running IPS code: 7.3(2)E4. The IPS that did not work had 7.1x on them. I upgraded my other IPS and now they all work.

I also used the new pySDEE code.

I am running Splunk 6.0.2

0 Karma

kdick
Explorer

dshpritz thank you for the answer! it has a few issues however. http://answers.splunk.com/answers/105193/cisco-ips-error-errno-8/135759 has an indentation error.
Seanp this is the cause of your problem and the reason the sdeegetlog is not populating.

In addition, not all cisco IPS SDEE servers run TLSv1, I had to set mine to SSLv3. Go to the cisco sdee server in your browser to check which version of ssl is needed.

After fixing the indentation errors (copy paste issue perhaps?) and changing the manual SSL input to ssl_version=ssl.PROTOCOL_SSLv3 I was able to connect succesfully!

seanp go ahead and try this and see if you can get it working.

Dshspritz, can you edit your answer to correct the indentation? I will upload what is currently working for me, hopefully I dont' encounter the same indentation errors from copy paste into this splunk answers site.

# The section below is to override the default socket connection
# which will fail with these devices. The newer version of openssl
# in Python does not support the ciphers these devices would like to use
import httplib
from httplib import HTTPConnection, HTTPS_PORT
import ssl
import socket

    class HTTPSConnection(HTTPConnection):
            default_port = HTTPS_PORT

            def __init__(self, host, port=None, key_file=None, cert_file=None,
                         strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
                         source_address=None):
                HTTPConnection.__init__(self, host, port, strict, timeout,
                                        source_address)
                self.key_file = key_file
                self.cert_file = cert_file

            def connect(self):
                sock = socket.create_connection((self.host, self.port),
                                                self.timeout, self.source_address)
                if self._tunnel_host:
                    self.sock = sock
                    self._tunnel()
                self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=ssl.PROTOCOL_SSLv3)

    #now we override the one in httplib
    httplib.HTTPSConnection = HTTPSConnection
    # ssl_version corrections are done

hortonew
Builder

Can you paste the full contents of your pySDEE.py file to pastebin, and link here? I'll check for any issues with it. Also, did you restart splunkd after editing?

0 Karma

mathewboarman
Explorer

I installed the pySDEE.py file attached earlier, tried it with v1 and v3... I get the same fault with both.

ERROR - Exception thrown in sdee.get(): URLError:
ERROR - Attempting to re-connect to the sensor: 173.30.4.68
INFO - Checking for exsisting SubscriptionID on host: 173.30.4.68
INFO - SubscriptionID: sub-1-c0a4a321 found for host: 173.30.4.68
INFO - Attempting to connect to sensor: 173.30.4.68
INFO - Successfully connected to: 173.30.4.68

Any ideas welcome...
thanks

0 Karma

hortonew
Builder

This fix worked for me. Exact troubleshooting steps + fix have been documented here: http://blog.hortonew.com/splunk-ciscoips-app-no-longer-pulls-from-ips

kdick
Explorer

Hi Sean,

The indent error is coming from these lines:

1 if self._tunnel_host:
2 self.sock = sock
3 self._tunnel()

It should be:
1 if self._tunnel_host:
2 self.sock = sock
3 self._tunnel()

Python will throw an error on the first one because it is expecting an indentation after the if.

The code I had in my post will not work for you as I have hardcoded SSLv3 and you need TLSv1. I have uploaded the full PSYDEE script that should work for you onto pastebin: http://pastebin.com/jCdhjHED

Please try and replace your pySDEE.py file with that. Let me know how that works.

seanp
Path Finder

kdick and dshpritz, thanks for the replies. Unfortunately the only the only way I have gotten this to work is by editing the $SPLUNK_HOME\Python-2.7\Lib\httplib.py library and adding the ssl_version:

self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=ssl.PROTOCOL_TLSv1)

I have tried modifying the indents (tabs vs space) and new line character (LF vs CRLF). Could you expand on the issue with indents? Mind sharing the start of the $SPLUNK_HOME\etc\apps\Splunk_CiscoIPS\bin\pysdee\pySDEE.py file? Unfortunately Python is not one of my scripting languages

0 Karma

kdick
Explorer

Sorry for posting a new answer! I don't have enough Karma to comment apparently ¯_(ツ)_/¯

0 Karma

dshpritz
SplunkTrust
SplunkTrust

A potential fix for this:

Take the code below, and paste it into the bin/pysdee/pySDEE.py file, at the top, right after the stock import statements:

# The section below is to override the default socket connection
# which will fail with these devices. The newer version of openssl
# in Python does not support the ciphers these devices would like to use
import httplib
from httplib import HTTPConnection, HTTPS_PORT
import ssl
import socket

class HTTPSConnection(HTTPConnection):
    "This class allows communication via SSL."
    default_port = HTTPS_PORT

    def __init__(self, host, port=None, key_file=None, cert_file=None,
        strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
        source_address=None):
        HTTPConnection.__init__(self, host, port, strict, timeout,
            source_address)
        self.key_file = key_file
        self.cert_file = cert_file

    def connect(self):
        "Connect to a host on a given (SSL) port."
        sock = socket.create_connection((self.host, self.port),
            self.timeout, self.source_address)
        if self._tunnel_host:
            self.sock = sock
            self._tunnel()
        # this is the only line we modified from the httplib.py file 
        # we added the ssl_version variable 
        self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=ssl.PROTOCOL_TLSv1) 

#now we override the one in httplib 
httplib.HTTPSConnection = HTTPSConnection 
# ssl_version corrections are done 

dshpritz
SplunkTrust
SplunkTrust

I did, and it was tested in another place. In both places, this was using this app: http://apps.splunk.com/app/528/ for the event collection.

0 Karma

seanp
Path Finder

dshpritz, thank you for the post, however it is not working for me in Splunk 6.0.3. With the added code in pySDEE.py as you describe the sdee_get.log no longer records anything success or failures. Did you get this working with the Cisco IPS app?

timpet
New Member

seanp: Could you post how to setup that lightweight forwarder so that it will work with the IPS.

0 Karma

timpet
New Member

Thanks a bunch. I was able to get this working once doing it through the GUI. You saved me a ton of time!

0 Karma

seanp
Path Finder

You should see the IPS logs are populating the following file:

$SPLUNK_HOME\etc\apps\Splunk_CiscoIPS\var\log\ips_sdee.log.

If you are getting that far, it would be in your forwarder config. You can always re-enable the GUI if you are more comfortable doing that than the .conf files. Possibly look at another forwarder and copy and paste.

0 Karma

timpet
New Member

Thanks for this. I think I have done everything from your posts.... on the new forwarding server I now see "- Successfully connected to: 10.x.x.x" in the sdee_get.log

I am at least half way there now 🙂

I think I must have something wrong in the forwarding setup. From what I can tell if I don't use my own CA certs then I just remove the bottom 4 lines from the output.conf file. Here is mine:

[tcpout]
defaultGroup = MAINSPLNK.DOM.com_9997
[tcpout: MAINSPLNK.DOM.com_9997]
server = MAINSPLNK.DOM.com_9997
[tcpout-server://MAINSPLNK.DOM.com_9997]
compressed = true

0 Karma

seanp
Path Finder

$SPLUNK_HOME/etc/system/local/outputs.conf
[tcpout]
defaultGroup = MyIndexer.MyDomain.com_9997

[tcpout:MyIndexer.MyDomain.com_9997]
server = MyIndexer.MyDomain.com:9997

[tcpout-server://MyIndexer.MyDomain.com:9997]
compressed = true
sslCertPath = $SPLUNK_HOME\etc\auth\MyForwarderPrivateKey.pem
sslPassword =
sslRootCAPath = $SPLUNK_HOME\etc\auth\MyRootCAPublicKey.pem
sslVerifyServerCert = true

Your outputs.conf file may appear different if you do not use your own CA certs. Let me know if you have questions.

0 Karma

seanp
Path Finder

$SPLUNK_HOME/etc/system/local/inputs.conf (may be found in
$SPLUNK_HOME\etc\apps\Splunk_CiscoIPS\local)
[default]
host = MyHost

[monitor://$SPLUNK_HOME\etc\apps\Splunk_CiscoIPS\var\log\ips_sdee.log.MyIPS_IPAddress]
sourcetype = cisco_ips_syslog
source = SDEE
disabled = false
index=MyIndex

0 Karma

seanp
Path Finder

On the server to collect the IPS logs, I downloaded and installed the full version of Splunk 5.0.5 from http://www.splunk.com/page/previous_releases which will be used for the lightweight forwarder.

I then installed the IPS app through the GUI (just easier and encrypted the password)

Under Data inputs » Files & directories I disabled $SPLUNK_HOME/etc/apps/Splunk_CiscoIPS/var/log/ as I do not need to index anything on the local server. At that point I changed it to the lightweight forwarder which disables the GUI. Then configured the inputs and outputs files.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

This looks a whole lot like https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371 which seems to be a bug in OpenSSL when attempting to do TLS version renegotiation. The bug was fixed in OpenSSL upstream and in Debian / Ubuntu.

But, Splunk ships with its own version of OpenSSL. In Splunk 6.0.0 it seems to be OpenSSL 1.0.1e, which is likely affected by this issue.

Ther launchpad link above suggests some (very very very hackish) workarounds like updating python standard library files. I would personally open a support case w/ Splunk and in the meanwhile perhaps downgrade to Splunk 5.0.5, which has an older OpenSSL. Or, you could install a 5.0.5 forwarder just for your IPS app...

seanp
Path Finder

Thank you for your responses. In the end I setup a separate server as a Splunk 5.0.5 lightweight forwarder. After reviewing the link Masa sent and my own results running the OpenSSL command, I am unsure as to the exact cause. Perhaps its a combination. Regardless, hopefully the developers of the application will update it soon.

0 Karma
Get Updates on the Splunk Community!

Splunk Certification Support Alert | Pearson VUE Outage

Splunk Certification holders and candidates!  Please be advised of an upcoming system maintenance period for ...

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...