I'm getting ready to roll out Universal Forwarder on about 200 Windows servers.
What are my options if I wanted to upgrade the Universal Forwarder software at some point in the future? How can I change what type of information these servers can report (ie, add/remove different types of event logs)
I found documentation about deployment server, but it is extremely vague and I'm not sure this is the right tool for what I'm trying to do - "The deployment server is Splunk's tool for pushing out configurations, apps, and content updates to distributed Splunk instances. You can use it to push updates to any Splunk component: forwarder, indexer, or search head." What kind of updates? What kind of configurations? Not sure what this actually means without concrete examples of what it can do.
Other than that, I know there are scripted batch files that can be used with the Universal Forwarder. Is there any other way of remotely configuring what information the forwarders are able to send?
Essentially the deployment server can distribute apps to a Splunk instance's etc/apps directory - and so, whatever apps can do, that's what you can distribute. You cannot update the forwarder itself using the deployment server.
I can see how this can be seen as vague in a sense, because an app can be so many different things that it's hard to give one exact definition. Generally speaking the most common thing is for them to carry configuration files - you can see all configuration files that Splunk can use in
As Ayn said, you can't use the deployment server to update the forwarder itself. You'll have to use some other deployment tool for that; see the topic about performing a remote upgrade of forwarders in the Distributed Deployment Manual for information about the command-line options.
You can use the deployment server to update your forwarder configuration. There is an extended example in the Distributed Deployment Manual.
Consider using Active Directory with GPO for managed software with the MSI. Other option is to use Powershell to a remote install from a shared software location.