Getting Data In
Highlighted

Handshaking issue with deployment server

Communicator

Hi,

I am getting following error message on universal forwarder logs:

11-10-2013 17:43:38.750 +0530 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=notconnected
11-10-2013 17:43:46.141 +0530 ERROR HTTPClient - Should have gotten at least 3 tokens in status line, while getting response code. Only got 0.
11-10-2013 17:43:46.141 +0530 INFO HttpPubSubConnection - Secure HTTP POST failed: Unknown read error
11-10-2013 17:43:46.141 +0530 INFO HttpPubSubConnection - Could not obtain connection, will retry after=83 seconds.
11-10-2013 17:43:50.750 +0530 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not
connected

0 Karma
Highlighted

Re: Handshaking issue with deployment server

Super Champion

What do you have in your deploymentclient.conf on the forwarder?

Highlighted

Re: Handshaking issue with deployment server

Path Finder

I'm getting the error too.

But only for deployment clients in a particular network zone.

I can telnet to the deployment server on TCP 8089 fine, but the clients get the errors above.

At this stage I think it is a routing issue, our firewall team has been involved but have not detected any drops.

C.

0 Karma
Highlighted

Re: Handshaking issue with deployment server

Engager

make sure the port (8089 in my case) is open

Highlighted

Re: Handshaking issue with deployment server

Splunk Employee
Splunk Employee

Check also make sure the local firewall / iptables is permitting TCP8089 to the DS host, and since these are different zones also confirm that the actual clients can connect to 8089 and not just your machine.

After that, make sure there is twoway (inbound / outbound) traffic through the firewall / acl for 8089 to the DS enabled.

Highlighted

Re: Handshaking issue with deployment server

Communicator

We had exactly the same issue with the same error message and we struggled to figure it out - this turns out to be a MTU setting issue with a data center switch. Makes sense, given the ability to telnet to a port, but the web service then fails to work.

0 Karma
Highlighted

Re: Handshaking issue with deployment server

Path Finder

I resolved the issue by flushing iptables,hope it can resolve your issue too.

iptables -F

cheers

Highlighted

Re: Handshaking issue with deployment server

Engager

Hi I can confirm that iptables -F works in a homelab.

0 Karma
Highlighted

Re: Handshaking issue with deployment server

Splunk Employee
Splunk Employee
0 Karma
Highlighted

Re: Handshaking issue with deployment server

Splunk Employee
Splunk Employee

This answer is unlikely to help in most cases, however, I was getting this error on my local laptop (lab) where I had Splunk Enterprise (Deployment server) and Splunk Universal Forwarder (UF) running with the UF's targetUri setting in deploymentclient.conf pointing to localhost (local machine's IP actually). The issue of course was they were both using 8089 for mgmt port. By changing the port on my Enterprise instance to 8091 and restarting the enterprise instance running the deployment server, issue was resolved. Use
./splunk set splunkd-port 8091 on my DS
Restart DS instance

0 Karma