After disabling default port 8089 due to app conflict, why is our universal forwarder unable to contact the deployment server?

[psutton_et]

We found that we have a new server type with a new internal application that is using port 8089 and would not allow the Splunk process to start. After several discussions about whether this port was needed to contact the deployment server we decided it was not and disabled the port as follows:

C:\Program Files\SplunkUniversalForwarder\etc\system\local\server.conf and added this stanza:

[httpServer]
disableDefaultPort = true

We were then able to start the Splunk Process, but it does not contact the deployment server and pull down its apps. I used the below link to do the research and have tried everything to get the Universal Forwarder to contact the Deployment server with no luck.
http://answers.splunk.com/answers/115242/can-someone-please-explain-to-me-why-splunk-universal-forwa...

The log file reports:

03-17-2015 10:49:51.163 -0600 INFO  DC:DeploymentClient - target-broker clause is missing.
03-17-2015 10:49:51.163 -0600 WARN  DC:DeploymentClient - Property targetUri not found. DeploymentClient is disabled.
03-17-2015 10:49:51.163 -0600 INFO  DS_DC_Common - Deployment Client not initialized.
03-17-2015 10:49:51.163 -0600 INFO  DS_DC_Common - Deployment Server not available on a dedicated forwarder.

I have the deployment configuration as follows:
C:\Program Files\SplunkUniversalForwarder\etc\apps\config_base\local\deploymentclient.conf

[deployment-client]
phoneHomeIntervalInSecs = 300

[target-broker:deploymentServer]
targetUri=xxxxxx.xx.xxx:8089

What am I missing?

[wrangler2x]

8089 is the port on the deployment server that the deployment clients (forwarders) connect to to do a phone home and to check if there are new or changed deployment apps that need to be downloaded. Turning off that port breaks the forwarder. If you really cannot use that port, you'll have to modify web.conf file in $SPLUNK_HOME/etc/system/local on the Deployment Server and add a [settings] stanza (or update the one that is already there -- if there is one already there) to have mgmtHostPort = 127.0.0.1:8089. This is on the Deployment Server, and change 8089 to some other port.

The problem with doing this is now you'll have to modify every deployed forwarder to look to that new port. Might be easier to change the new internal application! But if you are bent on doing that, edit $SPLUNK_HOME/etc/system/local/deploymentclient.conf on each (and every) forwarder to have this:

[target-broker:deploymentServer]
targetUri=xxx.xxx.xxx:8089

Again, you'll change 8089 to whatever port you chose to use on the Deployment Server.

[dflodstrom]

This shouldn't be the case. Based on what I've read you should be able to disable 8089 on your UF without affecting its role as a deployment client. Can you confirm that there isn't a conflict with $SPLUNK_HOME/etc/system/local/deployment client.conf ?

[cpetterborg]

You have disabled the default port for the HTTP server (the web interface), not the deployment server.

You need to set the deployment server to be on a particular port (we use 9089), and then set the UF deploymentclient.conf file to use that port instead of 8089 (your file above still is using 8089). The two configs should be the same port number. Our UF deploymentclient.conf file looks like this:

[deployment-client]
phoneHomeIntervalInSecs = 300

[target-broker:deploymentServer]
targetUri=xxx.xxx.xxx:9089

the etc/system/local/web.conf file for the deployment server has the following:

[settings]
httpport = 9000
mgmtHostPort = 127.0.0.1:9089
startwebserver = 1

I believe that this is the file that tells it which port to listen on. You can set it through the web UI.

[psutton_et]

Your answer is telling me to change the port on our entire env. We've been installing for over a year. This would be a huge endeavor. I was just trying to turn off the management port on the Universal Forwarders with the port 8089 confict. Your answer doesn't answer my question.

Can that be done. Can I turn off the management port on the Universal Forwarder and still communicate with the Deployment server?

disableDefaultPort = true|false
* If true, turns off listening on the splunkd management port (8089 by default)

[lakshman239]

If you want to change the mgmt port (8089) to another port on any Universal forwarder, yes, you can do that using web.conf on that server or via CLI.

For the UF to communicate to the Deployment server, ensure your deploymentclient.conf has the correct host/Ip and the mgmt port of the deployment server (usually 8089).

This way you don't need to change entire estate.