on tuesday i upgraded my deployment server, my searchhead and my 3 indexers from 4.1.6 to 4.2. I have not updated my LWFs. on my searchhead, indexers and LWF i now see the following error:
03-22-2011 16:28:50.591 WARN DeploymentClient - Unable to send phonehome message to deployment server. Error status is: rejected
the servers can connect to the deployment server listening port so it doesn't appear to be a network issue:
[splunk@indexer01~]$ telnet search01 8090 Trying x.x.x.x... Connected to search01 (x.x.x.x). Escape character is '^]'. ^]
how do I fix this? are there any debug logs i can turn on/look at to get a better idea of what exactly is failing?
i also installed the 4.2 universal forwarder on a couple hosts and they also get the same error when trying to connect to the deployment server.
support noted that in my deployment sever splunkd_access.log I have lots of lines like this: "POST /services/broker/phonehome/ HTTP/1.0" 404 200 - - - 41ms. which means my deployment clients are trying to POST to /services/broker/phonehome/, but are getting a 404 (file not found) response. still digging into what that means and how its happening.
I have noticed that the 4.2 Forwarder license does not seem to allow Deployment Server to run:
$ ./splunk list deploy-clients Current license does not allow the requested action $ ./splunk reload deploy-server Current license does not allow the requested action
On the logs of a client:
WARN DeploymentClient - Unable to send phonehome message to deployment server. Error status is: rejected
i forgot to update this with the solution, but jason is right. the 4.2 forwarder license does not allow the deployment server to run. I had to add the deployment server to my 4.2 license pool and it started successfully after that.