We have a requirement to forward logs from clients (Splunk universal Forwarders) to a server using SSL (tls1.2)
First Try: We installed same server certificate on both server and clients (as mentioned in the examples in splunk documentation and in splunk blogs). It worked fine.
Change request: Each client should have its own client certificate.
Second Try: We created multiple client certificates, one for each client. Installed those certificates on the client. We started getting a error: connection is not established.
For second try, we followed the below mentioned steps
Step 1: Created a CA Certificate - CACert.pem
Step 2: Created a Server Certificate using the above CA Certificate - ServerCert.pem
Step 3: Created four client certificates using the above CA Certificate - Client1Cert.pem, Client2Cert.pem, Client3Cert.pem, and Client4Cert.pem
Step 4: Installed certificates
Step 5: Restarted Splunk on server and clients.
...... Started seeing connection related errors in splunkd.log ............
How will universal forwarder clients validate that the server certificate that is presented is valid? Similarly, how will the server validate that the client certificate that is presented is valid?
What is wrong here? Could you please help.
As per this link , setting "requireClientCert = true" would require the following conditions to be met :
a) "rootCA" must point to a file containing the CA's public key.
b) The forwarder's server certificate defined by "sslCertPath" in outputs.conf is signed by that CA.
c) The forwarder has the password to read its own certificate ("sslPassword" in outputs.conf).
In our case, we were meeting all the conditions but still we faced issues.
In the same link, there is point which says -- "The purpose of this requireClientCert=true is to ensure that only forwarders that you have distributed a signed certificate to can connect to this indexer."
So, here is my observation:
requireClientCert = true should be set, when we are using same signed certificate on the server(receiver) and on all the clients (forwarders)