Why did the KVstore fail to start after using a more secure cipher in server.conf?

[daniel_splunk]

The splunk version is 7.1.0.
Here is the default cipher from server.conf

[sslConfig]
sslVersions = tls1.2
sslVersionsForClient = tls1.2
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-
SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-
AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256
ecdhCurves = prime256v1, secp384r1, secp521r1

Once I changed the cipher by removing all the non-ECDHE like this, kvstore failed to start.

cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-
 SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-
 AES128-SHA256:ECDHE-RSA-AES128-SHA256

mongod.log.

 2018-08-29T06:56:25.541Z I NETWORK  [thread1] connection accepted from 127.0.0.1:50090 #786 (1 connection now open)
 2018-08-29T06:56:25.542Z E NETWORK  [conn786] SSL: error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher
 2018-08-29T06:56:25.542Z I -        [conn786] end connection 127.0.0.1:50090 (1 connection now open)

[daniel_splunk · Accepted solution]

In splunk (starting from 6.5.2 onwards), the MongoDB (KVStore Engine) respect the SSL settings in Server.conf. However, the MongoDB doesn't support the more secure Cipher suites such as ECDHE.

If you check the mongo process by "ps -ef|grep mongo", you will see "--sslCipherConfig" is used to pass the cipher to mongo.

Currently, the only workaround is to add the cipherSuite = AES256-GCM-SHA384 to the cipherSuite of server.conf to workaround this.

[adckia]

Thanks for the insight. When, i.e. in which Splunk version, will this be solved?

[keio_splunk]

ECDHE ciphers support has been added to Splunk version 7.2.11 / 7.3.5 / 8.0.1 onward.

[daniel_splunk]

The root cause is due to mongo db can't support DH parameter.

https://jira.mongodb.org/browse/SERVER-24897

According to above mongo JIRA, it is fixed in 3.5.12

I believe 7.2.x will resolve this as 7.2.x is using 3.6.7 mongo db. However, you need to confirm this.

Splunk 7.2.0 mongo db version.
 2018-12-02T03:26:38.893Z I CONTROL  [initandlisten] db version v3.6.7-splunk

[chris_barrett]

FWIW we found that we still had to add AES256-GCM-SHA384 into the cipherSuite setting in server.conf in our 7.3.3 deployment.

[mstjohn_splunk]

Hey @daniel_splunk, thanks for updating us on this issue. I'm going to go ahead and accept your answer.