Knowledge Management

Why did the KVstore fail to start after using a more secure cipher in server.conf?

daniel_splunk
Splunk Employee
Splunk Employee

The splunk version is 7.1.0.
Here is the default cipher from server.conf

[sslConfig]
sslVersions = tls1.2
sslVersionsForClient = tls1.2
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-
SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-
AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256
ecdhCurves = prime256v1, secp384r1, secp521r1

Once I changed the cipher by removing all the non-ECDHE like this, kvstore failed to start.

cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-
 SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-
 AES128-SHA256:ECDHE-RSA-AES128-SHA256

mongod.log.

 2018-08-29T06:56:25.541Z I NETWORK  [thread1] connection accepted from 127.0.0.1:50090 #786 (1 connection now open)
 2018-08-29T06:56:25.542Z E NETWORK  [conn786] SSL: error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher
 2018-08-29T06:56:25.542Z I -        [conn786] end connection 127.0.0.1:50090 (1 connection now open)
1 Solution

daniel_splunk
Splunk Employee
Splunk Employee

In splunk (starting from 6.5.2 onwards), the MongoDB (KVStore Engine) respect the SSL settings in Server.conf. However, the MongoDB doesn't support the more secure Cipher suites such as ECDHE.

If you check the mongo process by "ps -ef|grep mongo", you will see "--sslCipherConfig" is used to pass the cipher to mongo.

Currently, the only workaround is to add the cipherSuite = AES256-GCM-SHA384 to the cipherSuite of server.conf to workaround this.

View solution in original post

daniel_splunk
Splunk Employee
Splunk Employee

In splunk (starting from 6.5.2 onwards), the MongoDB (KVStore Engine) respect the SSL settings in Server.conf. However, the MongoDB doesn't support the more secure Cipher suites such as ECDHE.

If you check the mongo process by "ps -ef|grep mongo", you will see "--sslCipherConfig" is used to pass the cipher to mongo.

Currently, the only workaround is to add the cipherSuite = AES256-GCM-SHA384 to the cipherSuite of server.conf to workaround this.

adckia
New Member

Thanks for the insight. When, i.e. in which Splunk version, will this be solved?

0 Karma

keio_splunk
Splunk Employee
Splunk Employee

ECDHE ciphers support has been added to Splunk version 7.2.11 / 7.3.5 / 8.0.1 onward.

Tags (3)
0 Karma

daniel_splunk
Splunk Employee
Splunk Employee

The root cause is due to mongo db can't support DH parameter.

https://jira.mongodb.org/browse/SERVER-24897

According to above mongo JIRA, it is fixed in 3.5.12

I believe 7.2.x will resolve this as 7.2.x is using 3.6.7 mongo db. However, you need to confirm this.

Splunk 7.2.0 mongo db version.
 2018-12-02T03:26:38.893Z I CONTROL  [initandlisten] db version v3.6.7-splunk
0 Karma

chris_barrett
Communicator

FWIW we found that we still had to add AES256-GCM-SHA384 into the cipherSuite setting in server.conf in our 7.3.3 deployment.

0 Karma

mstjohn_splunk
Splunk Employee
Splunk Employee

Hey @daniel_splunk, thanks for updating us on this issue. I'm going to go ahead and accept your answer.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...