topic Re: Handshaking issue with deployment server in Getting Data In

Handshaking issue with deployment server

rameshlpatel — Mon, 28 Sep 2020 15:14:36 GMT

Hi,

I am getting following error message on universal forwarder logs:

11-10-2013 17:43:38.750 +0530 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
11-10-2013 17:43:46.141 +0530 ERROR HTTPClient - Should have gotten at least 3 tokens in status line, while getting response code. Only got 0.
11-10-2013 17:43:46.141 +0530 INFO HttpPubSubConnection - Secure HTTP POST failed: Unknown read error
11-10-2013 17:43:46.141 +0530 INFO HttpPubSubConnection - Could not obtain connection, will retry after=83 seconds.
11-10-2013 17:43:50.750 +0530 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected

Re: Handshaking issue with deployment server

lukejadamec — Sun, 10 Nov 2013 13:51:19 GMT

What do you have in your deploymentclient.conf on the forwarder?

Re: Handshaking issue with deployment server

Conradj — Tue, 26 Nov 2013 19:44:30 GMT

I'm getting the error too.

But only for deployment clients in a particular network zone.

I can telnet to the deployment server on TCP 8089 fine, but the clients get the errors above.

At this stage I think it is a routing issue, our firewall team has been involved but have not detected any drops.

Re: Handshaking issue with deployment server

WedbushITOps — Fri, 18 Jul 2014 19:09:55 GMT

make sure the port (8089 in my case) is open

Re: Handshaking issue with deployment server

esix_splunk — Mon, 21 Jul 2014 05:09:56 GMT

Check also make sure the local firewall / iptables is permitting TCP8089 to the DS host, and since these are different zones also confirm that the actual clients can connect to 8089 and not just your machine.

After that, make sure there is twoway (inbound / outbound) traffic through the firewall / acl for 8089 to the DS enabled.

Re: Handshaking issue with deployment server

ebailey — Tue, 15 Sep 2015 16:46:02 GMT

We had exactly the same issue with the same error message and we struggled to figure it out - this turns out to be a MTU setting issue with a data center switch. Makes sense, given the ability to telnet to a port, but the web service then fails to work.

Re: Handshaking issue with deployment server

neelamsantosh — Tue, 27 Oct 2015 06:31:14 GMT

I resolved the issue by flushing iptables,hope it can resolve your issue too.

iptables -F

cheers

Re: Handshaking issue with deployment server

bohanlon_splunk — Sat, 30 Jan 2016 22:47:39 GMT

Check out the answer in this post:
https://answers.splunk.com/answers/217/error-httpclient-should-have-gotten-at-least-3-tokens-in-status-line-while-getting-response-code-only-got-0-what-does-it-mean.html

Re: Handshaking issue with deployment server

bmcclary_splunk — Sat, 27 May 2017 14:21:15 GMT

This answer is unlikely to help in most cases, however, I was getting this error on my local laptop (lab) where I had Splunk Enterprise (Deployment server) and Splunk Universal Forwarder (UF) running with the UF's targetUri setting in deploymentclient.conf pointing to localhost (local machine's IP actually). The issue of course was they were both using 8089 for mgmt port. By changing the port on my Enterprise instance to 8091 and restarting the enterprise instance running the deployment server, issue was resolved. Use
./splunk set splunkd-port 8091 on my DS
Restart DS instance

Re: Handshaking issue with deployment server

ebaileytu — Sat, 27 May 2017 18:07:13 GMT

That is really interesting since I am running the UF on the same host as the search head but I also have the management port for the UF turned off

[httpServer]
disableDefaultPort = true

and it is talking to another server for its deployment information.

I am going to turn the UF off for a bit to see if it helps.

Thanks!

Re: Handshaking issue with deployment server

ebaileytu — Tue, 29 Sep 2020 14:15:56 GMT

another common error message from the captain for each member that stops running scheduled searches

05-27-2017 15:02:32.502 -0500 ERROR SHCMasterArtifactHandler - failed on handle async replicate request sid=scheduler_adminxxxxRMD52a88c92ed83e8b0e_at_1495915200_11776_2E1C054F-9A8B-4D4A-BBC0-29F0562C7AED err='targetPeer="member", targetGuid="88275523-AE18-4CD9-AD67-7956E06449C1" cannot be valid target for artifactId=scheduleradmincaptain_RMD52a88c92ed83e8b0e_at_1495915200_11776_2E1C054F-9A8B-4D4A-BBC0-29F0562C7AED srcPeer="captain", srcGuid="A6A2F1D5-37C9-419C-A85E-A42376EDD483" reason="peer already has artifact"'

Re: Handshaking issue with deployment server

ohulea — Thu, 12 Dec 2019 15:34:41 GMT

Hi I can confirm that iptables -F works in a homelab.