I would like to secure splunkd (port 8089) on Splunk Universal Forwarders by using a throwaway self-signed certificate.
I tried the following methods:
1) Using msiexec to install Splunk Universal Forwarder, and also include the throwaway certificate for the forwarders
msiexec.exe /i splunkforwarder-<version>.msi DEPLOYMENT_SERVER="<deployment_server>:8089" AGREETOLICENSE=Yes CERTFILE=<throwaway forwarder certificate>.pem CERTPASSWORD=<private key password> /quiet
This method will install Splunk Universal Forwarder, and add the certificate into
$SPLUNK_HOME\etc\auth. However, after installation, it still uses the default Splunk certificate in
2) Deploy an app containing
server.conf to the deployment clients
[sslConfig] serverCert = $SPLUNK_HOME\etc\apps\ssl_app\cert\<throwaway forwarder certificate>.pem sslPassword = <private key password> sslVersions = tls
I understand this method does not work, as the configuration in
$SPLUNK_HOME\etc\system\local\server.conf will replace any configuration done in the app.
May I know the following:
a) What is the best way to configure Splunk Universal Forwarders to use a self-signed certificate for splunkd during installation?
b) What is the best way to configure Splunk Universal Forwarders to use a self-signed certificate for splunkd after installation?
If you name the files exactly like Splunk does, it will work.
If your source files are in C:\temp for example, use:
msiexec.exe /i splunkforwarder... CERTFILE=C:\temp\server.pem ROOTCACERTFILE=C:\temp\cacert.pem ... /quiet
If they are called server.pem and cacert.pem respectively, they will overwrite the default splunk-generated ones.
Passwords is where this gets interesting. You have couple options:
My preferred method is to give installer CERTFILE=C:\temp\server.pem with encrypted priv key and omitting CERTPASSWORD entirely. You'll want to stop splunk from launching with LAUNCHSPLUNK=0 so that system\local\server.conf isn't generated yet. This gives you opportunity to replace splunk.secret with your own (known) version and copy/paste your encrypted sslPassword. This way, neither your priv key nor cleartext password is ever revealed to whoever runs the installer script.
To answer your second question:
If splunk finds server.conf in one of the apps BEFORE fist launch (hence the importance of LAUNCHSPLUNK=0), it will NOT create system\local\server.conf. I took advantage of this and created few apps that get copy/pasted alongside the install. You'll have to be careful with naming them because of settings precedence, but crafted correctly, you can create your own defaults that live in apps and have system/local completely empty.
Name your defaults app zzzSystemLocalReplacement. Leave this app unmanaged by DS. Because it starts with "zzz" it will be matched as a last resort. Then, you can use deployment server to push down different app with new SSL certs when the time comes. New SSL cert app will then take higher precedence and become your effective configuration.
This procedure has few key advantages: