topic Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd in All Apps and Add-ons

Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

DimasSouza — Mon, 28 Sep 2020 20:23:03 GMT

Hello Community,

since I enabled the setting "requireClientCert = true" on our server.conf files the App "Splunk_TA_jmx" just stopped working. I pasted the error messages at the end.
Once the setting is returned to "false" the app starts working again.
We are using selfsigned Certificates on our Splunk to Splunk communications, apart from this App, all other connections are working perfectly with requireClientCert = true .

I even tried generating the file mx4j.ks. No success. 😞

I seems the App internal connection to splunk are being blocked, but I can't find a way to provide it with out certificates.

Any recommendation? Is it a bug?

We are running on SLES 11, Splunk 6.2.2 build 255606. Splunk Add-on for Java Management Extensions 3.0.0 (sandbox version is 3.0.1) and Oracle Java 1.8.

Thanks in advance,
Dms

on splunkd.log
06-26-2015 15:09:37.491 +0200 WARN  HttpListener - Socket error from 127.0.0.1 while idling: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate

on jmx.log
2015-06-26 14:26:09,630 - com.splunk.modinput.ModularInput -0    [main] ERROR  - Error executing modular input : Received fatal alert: handshake_failure : java.lang.RuntimeException: Received fatal alert: handshake_failure

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

jcoates_splunk — Mon, 29 Jun 2015 02:24:30 GMT

Java 8 means TLS is required, SSL won't work. I'd also upgrade to the latest Splunkd.

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

DimasSouza — Mon, 29 Jun 2015 11:13:16 GMT

That is not the issue here. A simple test with Java 1.8 and openssl s_server using the same certificates from my Splunk system returns successfull connections. This error can be reproduced by not sending a client certificate.
Btw. an updated version of splunk (sandbox running 6.2.3) returns the same problem.

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

jcoates_splunk — Mon, 29 Jun 2015 15:39:26 GMT

allow me to clarify -- this is the tested and supported connectivity matrix: http://docs.splunk.com/Documentation/AddOns/latest/JMX/Hardwareandsoftwarerequirements#Prerequisites

If you're trying to go outside of that, we don't think that it will work, but will happily accept being wrong if it comes with a support ticket and enhancement request, preferably with an example of how it was made to work 🙂

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

DimasSouza — Wed, 01 Jul 2015 13:25:57 GMT

We did not try to go outside of that. A Support ticket has just been opened (Case Nr. 251396).

If you wish to see it, just edit the [sslConfig] stanza of your server.conf as follows (alter paths and filenames as necessary) (either on a splunk server of a universal forwarder)
Once its done, restart splunk and check your splunkd.log and jmx.log files.

[sslConfig] 
allowSslCompression = false 
allowSslRenegotiation = false 
caCertFile = <self_signed_root_ca> 
caPath = <caPath> 
cipherSuite = TLSv1+HIGH:!SSLv2:!RC2:!RC4:!DES:!3DES:!MD5:!MD2:!EXP:!MEDIUM:!LOW:!PSK:!DSS:!aNULL:!eNULL:!SRP:!aECDH:!aECDSA@STRENGTH 
ecdhCurveName = prime256v1 
requireClientCert = true 
sslKeysfile = <sslKeysfile> 
sslKeysfilePassword = <sslKeysfilePassword> 
sslVersions = tls1.2

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

laserval — Wed, 01 Jul 2015 14:02:32 GMT

A simple test with Java 1.8 and openssl s_server using the same certificates from my Splunk system returns successfull connections.

Just for clarification; this means you have your certs imported in the Java keystore that is used by the app as well, correct?

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

DimasSouza — Mon, 03 Aug 2015 14:27:02 GMT

No for the following causes:
1- because there is no mention of it on the Installation steps ( http://docs.splunk.com/Documentation/AddOns/latest/JMX/Installationsteps)
2- The only point of conflict is the variable in question: requireClientCert = true (if it is set to off, the app starts working).

For me it is pretty clear that some improvement is due on this App.

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

i2sheri — Wed, 04 Nov 2015 08:50:33 GMT

it is a problem for python sdk too.
https://github.com/splunk/splunk-sdk-python/issues/123

Any solution for this problem without setting requireClientCert = false

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

i2sheri — Wed, 04 Nov 2015 08:50:56 GMT

Any solution or workaround for this problem without setting requireClientCert = false

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

DimasSouza — Wed, 04 Nov 2015 11:36:34 GMT

Hello Everybody,

here the official answer I got from Splunk support:
"Unfortunately the feedback from Dev is that JMX App does not support requireClientCert=true in server.conf.
They are planning to add the fix the one of the next releases of this App, so I would like to know if using requireClientCert=false it is a possibility based on your requirements"

So we have to work with requireClientCert=false for the time being.

Regards,
Dimas Souza

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

DimasSouza — Thu, 04 Aug 2016 08:10:26 GMT

Follow up:

This issue was included on the "Known Issues" list for this app with issue Numer: ADDON-5325

We're still waiting for a solution.

Re: Setting requireClientCert = true prevents "Splunk Add-on for Java Management Extensions" from communicating with splunkd

mshenoyp — Tue, 04 Oct 2016 13:03:09 GMT

Is this problem resolved? What is the fix for this?