Pushing self signed certificates to universal forwarders

[alanzchan]

Is there a Splunk recommended solution to pushing self signed SSL certificates to thousands of universal forwarders?

We tried bundling the certificates into an app and pushing it out to the universal forwarders. However, I believe that the default configurations set in /system/local/ on the universal forwarders will overwrite the configurations set within the app.

Is there a way around this or is there a better alternative solution?

[koshyk]

If you have settings in etc/system/local of the UF, then it is almost impossible to override. This is why any enterprise installation should NOT have any items in etc/system/local
The only way is to use an external tool like puppet/ansible which can do the re-installation for you.

There is any tricky way to do this (but quite dangerous). The idea is to PUSH an app with python or shell script code, which can delete entries (or do stuff at OS level) an restart splunk. But this is NEVER recommended for enterprise level systems, as you won't have any control if it goes wrong.

Whatever you did is correct of putting all your TLS/self-signed certificates into an app and manage it centrally and make it part of deployment-server.

[alanzchan]

Hi koshyk,

How I understand SSL, the server.conf and outputs.conf both need to be modified with the correct configurations. Once we accept the license on a UF, a server.conf will automatically be generated. I'm trying to override the /system/local server.conf file that was automatically generated with the server.conf that I have defined in my application.

[koshyk]

I can't find a way unfortunately other than with help of separate tool or doing the ugly solution as mentioned above

[alanzchan]

In your initial response, are you saying there should be absolutely no settings in /system/local? Not even server.conf? Isn't server.conf automatically generated when you install the universal forwarder and accept the license? Kind of confused here.

[koshyk]

I meant to say, as splunk admins, we shouldn't put configurations in /system/local. anything which is auto-generated is Okish, as beyond our control.