I'm having an issue with getting the Universal Splunk forwarder to reach out to the deployment server. I have 12 servers that I've configured all the same way and 8 of them are working properly but for some reason these last 4 will not reach out. It's not a firewall issue as I can telnet to 8089 to the deployment server without issue and all of the servers have an entry in the serverlist.conf file on the deployment server. In each server we are seeing this in the splunk.d logs
08-26-2020 11:18:32.341 -0500 DEBUG DC:DeploymentClient - Creating a DeploymentClient instance
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : disabled=false
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : workingDir=c:\Program Files\SplunkUniversalForwarder\var\run
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : clientName=DDF77B18-237A-4753-B250-BC8D91C28FF4
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : repositoryLocation=c:\Program Files\SplunkUniversalForwarder\etc\apps
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : serverRepositoryLocationPolicy=acceptSplunkHome
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : serverEndpointPolicy=acceptAlways
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : maxRetries=3
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : waitInSecsBetweenRetries=60
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : phoneHomeIntervalInSecs=60
08-26-2020 11:18:32.356 -0500 INFO DC:DeploymentClient - target-broker clause is missing.
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : endpoint=$deploymentServerUri$/services/streams/deployment?name=$tenantName$:$serverClassName$:$appName$
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - Setting : reloadDSOnAppInstall=false
08-26-2020 11:18:32.356 -0500 WARN DC:DeploymentClient - DeploymentClient explicitly disabled through config.
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 1
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 2
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 3
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 4
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 5
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 6
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 7
08-26-2020 11:18:32.356 -0500 DEBUG DC:DeploymentClient - trace 8
08-26-2020 11:18:32.356 -0500 INFO DS_DC_Common - Deployment Client not initialized.
08-26-2020 11:18:32.356 -0500 INFO DS_DC_Common - Deployment Server not available on a dedicated forwarder.
Our deployment.conf is in the correct place and it explicity has disabled set to false under the [deployment-client] heading. I've uninstalled and installed the forwarder multiple times and restarted the services on the deployment server. It just somehow thinks the Deployment client is disabled by default in the config on these 4 servers.
I think that those should be in deploymentclient.conf which is under .../apps/TA_your_deploymentclient/default directory.
Choose what ever name you want to your TA name.
r. Ismo
Hi
what you got with command
splunk btool deploymentclient list --debug
on those UFs?
r. Ismo
Should have probably specified I'm on Windows. It did give me some output but it also told me this
Command error: The subcommand 'deploymentclient' is not valid for command 'btool.exe'.
I saw it already on your logs and this should work also in windows. I haven’t any windows installation on my hands but maybe this helps. https://community.splunk.com/t5/Getting-Data-In/Can-not-see-the-output-of-btool-in-windows/td-p/4516...
r. Ismo
It doesn't return anything
C:\Program Files\SplunkUniversalForwarder\bin>splunk.exe btool deploymentclient list --debug
C:\Program Files\SplunkUniversalForwarder\bin>
Basically that means that you haven’t configured DS for this client. Can you check this also on UF which works with DS?
https://docs.splunk.com/Documentation/Forwarder/8.0.5/Forwarder/Configuretheuniversalforwarder
there are more information how to add DS to UF. The best solution is that you have separate app which contain that part and which is automatically installed together with UF.
r. Ismo
Thanks. Shouldn't that all come from the deployment.conf file I have setup during startup
[deployment-client]
disabled = false
sslVersions = tls1.2
sslVerifyServerCert = true
sslRootCAPath = $SPLUNK_HOME/etc/apps/<path to cert>
[target-broker:deploymentServer]
targetUri = <deployment server hostname>:8089
Our deployment looks for that file in $SPLUNK_HOME/etc/apps/<path to deployment.conf>
I think that those should be in deploymentclient.conf which is under .../apps/TA_your_deploymentclient/default directory.
Choose what ever name you want to your TA name.
r. Ismo
Thank you! It was all because of a typo in the filename. I had accidentally renamed it deployment.conf instead of deploymentclient.conf.