SSL Certificates for thousands of Clients

[inventsekar]

Hi,

1. after certificates created, how to push them to, lets say, ten thousand deployment clients?
2. someone said some python scripts are there to do this task, any suggestions please.
3. on some posts I read that we can use deployment server itself to push the certificates, can we follow this idea, pls suggest

4. approx. how long it will take to do this, thru python script and thru deployment server?

5. the important question is that - how to renew the certificates when they expire?
   we can create a new certificate. and lets say it takes one day to deploy the certificates on all 10 thousand hosts,

6. do I first install the certificate on indexer/ DS/ search head and then I will need to push the certificate to a client and once splunk service restarts, it will make the secure connection. is this correct?

7. lets say I installed the new certificate on DS/Indexers/Search heads and then if it takes 2 days to send the new certificates to all 10 thousand clients, meaning, this two days how the client can communicate with indexer/DS?

Please suggest, thanks.

[asimagu]

I think the CertNanny project is trying to automate some of the processes.
It may be worthy to have a look there too

https://github.com/certnanny

[inventsekar]

Thanks Asimagu, will check it.

[inventsekar]

sorry for asking again, but, can someone please update clearly about this -
can I have two SSL certificates deployed on a single indexer? if yes, on same port or different ports?

the issue is - during Certificates renewal,
we would like to follow this process -
1. install a renewed certificate on indexer (while the old SSL certificate is still deployed)
2. deploy the renewed certificate to forwarders (while some forwarders may be still having the old certificates)
3. the UF's which got the renewed certificates will start communicating with the indexer's renewed certificate.
4. whereas, the old UF's, until certificate renewal, will still be communicating with the indexer with indexer's old certificate.

is this possible? how to add two [SSL] stanza's on outputs.conf?

[SSL]
rootCA = $SPLUNK_HOME/etc/certs/cacert.pem
serverCert = $SPLUNK_HOME/etc/certs/splunk-idx-01.pem

[SSL]
rootCA = $SPLUNK_HOME/etc/certs/renewedcacert.pem
serverCert = $SPLUNK_HOME/etc/certs/renewedsplunk-idx-01.pem

[P86]

I would try this in stages...

this are just my thoughts about it.

1. Set up heavy forwarders that accept data inputs with the new certificate and output them with the old to you existing environment.
2. Replace the old certs (you hopefully configured them in a custom app like org_all_forwarderoutputs) and push them with your deployment server out to your forwarders. Make sure they are pointing now to the heavy forwarders.
3. Wait till all forwarders are updated. In this time indexers will accept the data from the not yet updated and heavy forwarders from the already updated.
4. Update your indexers to the new certs and also update the outputs from your heavy forwarders to the new certs
5. Deploy your org_all_forwarderoutputs again to your forwarders pointing directly to the indexers with the new certs
6. Delete the heavy forwarders after all forwarders switched back to the indexers.

I would highly recommend to test that in a smaller environment before...

[inventsekar]

Thanks P86, much appreciated, lets wait for other views and ideas.

[inventsekar]

There may be situations that some UF's are sending data to indexers, without HF. Hmm, not sure of how to handle this situation.
Suggestions please.

[P86]

what situations do you mean? Well what I was trying to explain was that in the transition phase they can send to the indexers with the old certs or to the HFWs with the new cert. This phase takes as long as you need to update them all.

[inventsekar]

Set up heavy forwarders that accept data inputs with the new certificate and output them with the old to you existing environment ///

would like know info about this one please.