Getting Data In

TcpOutputProc - The TCP output processor has paused the data flow

splunkcol
Builder

I open a new thread because in the previous one I was reviewing several errors at the same time

for this specific error message I have already read all the forum posts and the ones I have found on the internet but I am still having problems

• I have no Licence problems
• I don't have another output.conf file in the heavy forwarder
• I do not have a high performance or consumption in IOPS after consulting with the command iostats and iotop -

The devices through syslog send the logs to the heavy carrier that receives them and when reviewing them the logs are with the date and time updated all the time

For some reason the heavy forwarder doesn't constantly forward the logs to the indexers, as when querying with
index = * host = xxxx | statistics count per host _time

I see records but with delays of 8-10 hours.

When checking the logs in var / log / splunk / splunkd.log

I use grep xxxx splunkd.log to check the errors only from the host that interests me and that is where I see the message

09-21-2020 07:20:48.483 -0500 WARN TcpOutputProc - The TCP output processor has paused the data flow. Forwarding to host_dest=indexer inside output group default-autolb -group from host_src=xxxxx has been blocked for blocked_seconds=100. This can stall the data flow towards indexing and other network outputs. Review the receivin g system's health in the Splunk Monitoring Console. It is probably not accepting data.

splunkcol_0-1600702525385.png

the heavy forwarder does not have high IOPS problems

The problem seems to be in the indexers that are not able to receive the information.

In that order of ideas, what do you recommend reviewing?

Labels (2)
0 Karma

thambisetty
SplunkTrust
SplunkTrust

The servers configured in outputs.conf are not performing well. there could be many reasons:

  1. network issue from Heavy forwarder to Indexer
  2. indexers are overwhelmed with events coming in or busy in serving requests from search head.

check all servers (indexers) in outputs.conf of Heavy forwarder are healthy (CPU and memory utilization).

check if you have deployed outputs.conf to indexers by mistake. generally indexers don't have outputs.conf. 

 

————————————
If this helps, give a like below.
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...