I have about 110 forwarders that point to a central Splunk instance. All the forwarders are running 4.1.2 and the central is 4.1.3. Upgrading to 4.1.5 (latest) at the central point isn't a big deal, but to do it at 110 devices is. How do people upgrade the remote forwarders? I am about to go and set up FSChange to monitor specific EXE's and DLL's that run critical applications on each forwarder, so again I have to visit 110 devices.
I also tried to utilize DeploymentServer & DeploymentClient but these never worked. The information in the setup would show up, but no applications ever pushed.
How is everyone managing making changes across forwarders and upgrading? I am concerned as I read documentation about FSChange and setting up this monitoring, it is for 4.1.5 and the forwarders are behind that.
quite a lot of questions in one but let's see if i can give a general answer and cover them all:
How do you upgrade remote forwarders:
Use a script or a third party tool such as yum to tell the forwarders to grab the new installation file and upgrade splunk to the current version. A simple script that contains wget, tar and some moves and directory changes should suffice.
Making splunk configuration changes across remote machines should be done by deployment server and client connections. Make sure you make that work, for at least one machine, then use a script to enable it across your domain. Once the connection between server and clients has been confirmed you can add all your config files within an app (or multiple) under the deployment-apps on the Dep-Server, and then reload configs. This should suffice in the clients grabbing the latest config files.