How to configure all forwarders from an old deployment server to a new deployment server?

[abhay24]

We are migrating deployment-apps, Forwarders, from one Deployment server to another Deployment server.
In the process, I moved all deployment-apps to the new Deployment server, copied serverclass.conf also.

I could see all server classes and apps on forwarder management also, but the issue I'm having is how we can configure the forwarders to new deployment server?

We can do it through forwarder, but it's taking too much time and we don't have access to all those servers now.

So how can we change the deployment-client.conf for all the forwarders at the same time from our old/new deployment server?

[jpvlsmv]

You can... but it's ugly and error-prone.

The problem with deploying a deploymentclient.conf in an application is that the settings there are overridden by etc/system/local/deploymentclient.conf. So if you can change that (system/local) file, you're in business.

Ansible, Chef, Salt, Puppet, etc. are tools to change the file on the system, which is useful if they are already there, and you are allowed to make a change in the CM tool or can find a sysadmin long enough to explain what you need.

But you have Splunk on the system already, and we can do it in Splunk as a Splunk admin.

1) Create a deploy-client-config app in Splunk. You need 3 things in it (in addition to what comes out of the Blank application template):

• bin/remove_deploy_system_setting.[bat|py], a script that (re)moves $SPLUNK_HOME/etc/system/local/deploymentclient.conf and restarts splunk
• default/inputs.conf that runs the above script every... say 5 minutes
• default/deploymentclient.conf that points at the new DS

2) Use the old deployment server to push this out to everybody (restart splunk after)
3) Create a same-named app on the new deploy server that just has the default/deploymentclient.conf piece (not the script or inputs.conf)
4) Tell the new deploy server to install the new app

A future migration or DS change (such as new https keys) would only require deploying a new version of the "deploy-client-config" app.

--Joe

[masonmorales]

You can't. That's not a feature of deployment server, at least at the time I'm writing this. Most of us in large environments use a configuration management system (e.g. Ansible, Chef, Salt, Puppet) to change things like deploymentclient.conf across all of our forwarders.