Indexer cluster SSL migration from default SSL to self-signed with least downtime?

[inventsekar]

Hi,
1. Lets assume I have around 4 cluster peers with Splunk's default SSL. To migrate from Splunk's default SSL to self-signed SSL,
can I migrate the cluster peers one by one? I mean, on an indexer cluster, can I have two sets of SSL certificates (Splunk's default SSL and my own self-signed SSL)?
2. During the migration, the deployment server should be sending the new self-signed SSL certificates to forwarders. Is this possible?
I mean, one deployment server, handling two sets of SSL certificates.

[inventsekar]

Hi All, any ideas and suggestions about without using the 2nd port please.. as you know, on production systems it would be difficult to get 2nd port opened for this task alone..
any other ideas, suggestions please..

[hardikJsheth]

Do you want to use SSL certificates for encrypting communication between forwarder and indexer or you are referring to changing SSL certificates for Management port ?

With Splunk 6.3 and above it uses same certificates for all the nodes within indexer cluster including master node.

[inventsekar]

we want to use SSL certificates for encrypting communication between forwarder and indexer

[esix_splunk]

Theres no documented process for this, but thinking about a few different scenarios here, here's what I see as working without downtime..

General Outline--
1) Add a new splunktcp-ssl input on your indexers, via the cluster master, on a different port then your current port. E.g. 9998 instead of 9997. This should require a rolling restart to enable the config
2) Create a new app that has the new certs and outputs.conf to point to the splunktcp-ssl on 9998 on your indexer cluster
3) Use the DS to deploy this to clients, and remove the other outputs.conf

As clean up, you can validate that all of your clients are sending to the splunktcp-ssl input on your indexers. Once validated, you can disabled the the non-SSL port on the cluster, and copy the splunktcp-ssl config to 9997 with the same cert. You can then update the primary outputs.conf app on your DS and your clients will get updated and send to 9997.

[inventsekar]

Thanks Esix,.. any ideas and suggestions about without using the 2nd port?
on a indexer cluster, all cluster peers should have the SSL certificate(s) from same root CA, right

[inventsekar]

Hi Esix/All,

on an indexer cluster, can I have two sets of SSL certificates (Splunk's default SSL and my own self-signed SSL)?
lets assume i have an indexer cluster with 10 indexers. can i have 8 indexers with Splunk default SSL certificates and 2 indexers with my own self signed certificates? is that possible, please suggest.

[inventsekar]

SSL certificates migration process is not documented at all. also i am not seeing any posts related to this topic. Wondering how !!!