I have SSL/TLS enabled all comms between search head, deployment server, indexers and universal forwarders.
server.conf:
[sslConfig]
caCertFile = chain.pem
caPath = $SPLUNK_HOME\etc\auth\DOD
requireClientCert = false
sslKeysfile = password.pem
sslKeysfilePassword = **********
[kvstore]
caCertpath = $SPLUNK_HOME\etc\auth\DOD
sslKeysPassword = *********
sslKeysPath = $SPLUNK_HOME\etc\auth\DOD\private.pem
===============================================================================================
When I start Splunk servers I am getting errors in the mongod.log:
W CONTROL No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile parameter
E NETWORK cannot read certificate file: d:/splunk/etc/auth/DOD/private.pem error:0906D06C:PEM routines:PEM_read_bio:no start line
F CONTROL Failed global initialization: Location16778 ssl initialization problem
===============================================================================================
sslCAFile appears to be a mongod command line variable.
Is Splunk suppose to start and pass the sslCAFile variable somewhere???
Robert
On our servers the Old Splunk certs expired Oct 1 2015. We just upgraded to 6.3 and new certs were not generated. Recreating all certs with ./splunk createssl fixed it.
Still nothing. I am leaning towards the FIPS mode not being enabled when the server was built.
No matter what key I create or try to set in the [kvstore] area, I get the same errors about not being able to read the key that is set in the [sslconfig] for sslKeysfile. why does mongod/kvstore keep trying to read that password.pem key????
W CONTROL No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile parameter
E NETWORK cannot read certificate file: d:/splunk/etc/auth/DOD/password.pem error:0906D06C:PEM routines:PEM_read_bio:no start line
F CONTROL Failed global initialization: Location16778 ssl initialization problem
I deleted these lines and restarted and mongod starts without errors
[sslConfig]
caCertFile = chain.pem <--------------------------------deleted
caPath = $SPLUNK_HOME\etc\auth\DOD <--------------------------------deleted
requireClientCert = false <--------------------------------deleted
sslKeysfile = password.pem <--------------------------------deleted
sslKeysfilePassword = $1$**********
[kvstore] <--------------------------------deleted
caCertpath = $SPLUNK_HOME\etc\auth\cacert.pem <--------------------------------deleted
sslKeysPath = $SPLUNK_HOME\etc\auth\server.pem <--------------------------------deleted
sslKeysPassword = password <--------------------------------deleted
There's something wrong with my certs.... .I'll look into them more.
what was the resolution here?
command line: I did:
splunk stop
splunk createssl server-cert -d d:\splunk\etc\auth -n server
server.pem was created
my \splunk\etc\system\local\server.conf [sslconfig] and [kvstore] look like this:
[sslConfig]
caCertFile = chain.pem
caPath = $SPLUNK_HOME\etc\auth\DOD
requireClientCert = false
sslKeysfile = password.pem
sslKeysfilePassword = $1$**********
[kvstore]
caCertpath = $SPLUNK_HOME\etc\auth\cacert.pem
sslKeysPath = $SPLUNK_HOME\etc\auth\server.pem
sslKeysPassword = password
I did splunk start. there are no errors.
the sslKeysPassword stanza under kvstore, is now equal to $1$********** (matches the sslconfig hash)
mongod.log still giving same errors:
W CONTROL No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile parameter
E NETWORK cannot read certificate file: d:/splunk/etc/auth/DOD/password.pem error:0906D06C:PEM routines:PEM_read_bio:no start line
F CONTROL Failed global initialization: Location16778 ssl initialization problem
mongod/kvstore still keeps looking at the sslconfig area.....
Robert
You don't need the caCertpath and sslKeysPath for the kvstore stanza. Also make sure you recreate the web certs as I'm not sure which ones apply here. As for names of the certs I just went for "server" for the server certs to replace the defaults.
Btw your ca path may need to be changed to the default. Just uncomment that parameter
It looks like you need to add the actual file to the value specified in caCertpath in the [kvstore] stanza (e.g., caCertpath = $SPLUNK_HOME\etc\auth\DOD\chain.pem).
From the server.conf spec:
caCertPath =
* Public key of the signing authority.
* If specified, it will be used in KV Store SSL connections and
authentication.
* Must be specified if FIPS is enabled (i.e. SPLUNK_FIPS=1), otherwise, KV
Store will not be available.
* Only used when FIPS is enable
You could use btool to see what is actually being applied and what file it is reading it from:
$SPLUNK_HOME\bin\splunk btool server list --debug
I typo'd this in my original post: E NETWORK cannot read certificate file: d:/splunk/etc/auth/DOD/private.pem error:0906D06C:PEM routines:PEM_read_bio:no start line <--- shud be password.pem, not private.pem
I made these two new files root1 and root2 and changed the server.conf file:
caCertPath = $SPLUNK_HOME\etc\auth\DOD\root1.pem
sslKeysPassword = *********
sslKeysPath = $SPLUNK_HOME\etc\auth\DOD\root2.pem
root1.pem contains the public key for the root authority
root2.pem contains my password protected private key
Restarted splunk services. Still no good. mongod.log has same error. It's like the [kvstore] configs are not even being recognized:
W CONTROL No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile parameter
E NETWORK cannot read certificate file: d:/splunk/etc/auth/DOD/password.pem error:0906D06C:PEM routines:PEM_read_bio:no start line
F CONTROL Failed global initialization: Location16778 ssl initialization problem
What's this about FIPS ???? If we did not enable SPLUNK_FIPS = 1 in our initial server build, will all of this stuff not work????????