https://community.splunk.com/t5/Getting-Data-In/Open-TCP-Connections-from-forwarders-to-indexer/m-p/180412#M36180 <P>When watching the network traffic from a Splunk Universal Forwarder to a Indexer there appears to be two connections. </P> <P>There is a standard “data forwarding connection” (long lived, shows up with sourceIp and sourcePort in metrics.log under the tcpin_connections group) but there also appears to be a secondary connection opened by the forwarder. The second connection occurs every 30 seconds and lasts about 1 second after which it seemed to be closed by the forwarder.</P> <P>What is the second connection used for? Does it indicate a problem with Splunk?</P> Wed, 28 May 2014 07:25:11 GMT dshakespeare_sp 2014-05-28T07:25:11Z https://community.splunk.com/t5/Getting-Data-In/Open-TCP-Connections-from-forwarders-to-indexer/m-p/180412#M36180 <P>When watching the network traffic from a Splunk Universal Forwarder to a Indexer there appears to be two connections. </P> <P>There is a standard “data forwarding connection” (long lived, shows up with sourceIp and sourcePort in metrics.log under the tcpin_connections group) but there also appears to be a secondary connection opened by the forwarder. The second connection occurs every 30 seconds and lasts about 1 second after which it seemed to be closed by the forwarder.</P> <P>What is the second connection used for? Does it indicate a problem with Splunk?</P> Wed, 28 May 2014 07:25:11 GMT https://community.splunk.com/t5/Getting-Data-In/Open-TCP-Connections-from-forwarders-to-indexer/m-p/180412#M36180 dshakespeare_sp 2014-05-28T07:25:11Z https://community.splunk.com/t5/Getting-Data-In/Open-TCP-Connections-from-forwarders-to-indexer/m-p/180413#M36181 <P>The secondary connection is a "healthcheck" probe.<BR /> The forwarder uses this connection to evaluate whether the receiver still in a "good working condition". When a receiver is effectively stalled so it is not processing more data, or<BR /> when it is in the process of shutting down, the receiver closes the TCP socket. A forwarder which cannot connect to the indexer's incoming socket will put the indexer into quarantine and not associate any more data streams with that indexer until the health check works again. This<BR /> allows Splunk to try to complete outgoing data streams, and ensures that if data need to be resent (eg useACK timeout) it would not pointlessly select this indexer even if we are already<BR /> connected to it.</P> Wed, 28 May 2014 07:33:28 GMT https://community.splunk.com/t5/Getting-Data-In/Open-TCP-Connections-from-forwarders-to-indexer/m-p/180413#M36181 dshakespeare_sp 2014-05-28T07:33:28Z https://community.splunk.com/t5/Getting-Data-In/Open-TCP-Connections-from-forwarders-to-indexer/m-p/180414#M36182 <P>also see <A href="http://wiki.splunk.com/Community:Splunk_Socket_Behavior">http://wiki.splunk.com/Community:Splunk_Socket_Behavior</A></P> Wed, 28 May 2014 07:38:58 GMT https://community.splunk.com/t5/Getting-Data-In/Open-TCP-Connections-from-forwarders-to-indexer/m-p/180414#M36182 dshakespeare_sp 2014-05-28T07:38:58Z