Deployment Architecture
Highlighted

After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Champion

Hi,

I upgraded a Search Head to 6.6.0, and am getting the following messages continuously...

5-10-2017 13:12:10.558 -0400 WARN  SSLCommon - Received fatal SSL3 alert. ssl_state='SSLv3 read client hello C', alert_description='handshake failure'.
05-10-2017 13:12:10.558 -0400 WARN  HttpListener - Socket error from 127.0.0.1 while idling: error:1408A10B:SSL routines:ssl3_get_client_hello:wrong version number
05-10-2017 13:12:13.181 -0400 WARN  SSLCommon - Received fatal SSL3 alert. ssl_state='SSLv3 read client hello C', alert_description='handshake failure'.
05-10-2017 13:12:13.181 -0400 WARN  HttpListener - Socket error from 127.0.0.1 while idling: error:1408A10B:SSL routines:ssl3_get_client_hello:wrong version number
05-10-2017 13:12:15.624 -0400 WARN  SSLCommon - Received fatal SSL3 alert. ssl_state='SSLv3 read client hello C', alert_description='handshake failure'.
05-10-2017 13:12:15.624 -0400 WARN  HttpListener - Socket error from 127.0.0.1 while idling: error:1408A10B:SSL routines:ssl3_get_client_hello:wrong version number
Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Communicator

Its a bug ... Roll back to previous version and see

Everything works as normal

Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Ultra Champion

Do you see any communication failures or just these warnings?

Include any SSL related conf from your server.conf or web.conf (probably good to use btool here). Since it was an upgrade, there's always a chance there was a leftover ssl config from a prior release that conflicts with modern security requirements for SSL.

0 Karma
Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Engager

any updates on this? Im experiencing the same issue now.

0 Karma
Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Explorer

I'm also experiencing this after upgrade from 6.5.1 to 6.6.1. Had to rollback to previous version and all worked.

0 Karma
Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

SplunkTrust
SplunkTrust

Are you using older universal forwarders pre 6.2.x and sending traffic to a splunk tcp SSL port on the indexer?

In particular the older 6.0/6.1 series releases:
6.0.0 to 6.0.6 forwarders
6.1.0 to 6.1.4 forwarders

If so you can make the changes described in the known issues for 6.6.2 or upgrade your forwarders to a new version.

I suspect that your seeing older forwarders attempting to use an SSL/TLS cipher suite that is no longer supported by a modern version of the Splunk enterprise server.

0 Karma
Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Ultra Champion

Looks like there's some known issues related to SSL and upgrades.

Do any of these items seem like the cause? http://docs.splunk.com/Documentation/Splunk/latest/ReleaseNotes/Knownissues

Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Splunk Employee
Splunk Employee

Changes to the cipher suites between versions of splunk mean that OOTB the two versions of splunk will not have a common cipher to share the documentation advises providing a common cipher the two versions can agree on.

SSL/TLS are protocols - NOT ciphers. In particular, TLS is an evolution of SSL.

The relevant change is in $SPLUNK_HOM/etc/system/default/server.conf, and is the change to cipherSuite. In 6.4.1 this is set to

TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH

and in 6.6.1 this is set to

ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256

TLSv1+HIGH (and corresponding for TLSv1.2) means all ciphers compatible with TLSv1 of HIGH strength. There is some overlap here with the ciphers compatible with SSL3.0. However, none of the SSL3.0 ciphers appear in the 6.6.1 list.

To see this more clearly, take a Linux system with openssl installed (almost any Linux system will do!).

Run:

openssl ciphers SSLv3+HIGH
openssl ciphers TLSv1+HIGH

Note that these give you the same results. However, they all end with SHA. In the explicit list you provide in 6.6.1 they all end with SHA, so it's easy to see that there's no overlap with SSLv3+HIGH and the new list in 6.6.1 - leading to the behaviour observed. Any system (such as Splunk 6.1) which only supports TLS1.0 and below (including SSL3) won't be able to communicate with a Splunk 6.6.1 server with default config only suitable for TLS1.2.

Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Path Finder

thank you nclancy, this was a fantastic help.

0 Karma
Highlighted

Re: After upgrading to Splunk 6.6.0, why am I receiving warning messages such as "WARN SSLCommon - Received fatal SSL3 alert"?

Ultra Champion

@a212830 - Would you accept this answer if it helped?

0 Karma