Getting Data In

How to resolve "TcpOutputProc - Queue for group ICSRouting-checkpoint has begun dropping events" error?

sassens1
Path Finder

Hello,

We use a Heavy Forwarder (HF) to forward CheckPoint logs to an external third-party SIEM using the TCP protocol.
I have noticed from time to time this kind of errors:

01-25-2017 15:47:44.071 +0100 INFO TcpOutputProc - Queue for group ICSRouting-checkpoint has stopped dropping events
01-25-2017 15:47:44.688 +0100 WARN TcpOutputProc - Queue for group ICSRouting-checkpoint has begun dropping events

just a few milliseconds of failure?
I checked my queue size which seems ok:

02-08-2017 20:57:22.077 +0100 INFO Metrics - group=queue, ingest_pipe=0, name=tcpout_icsrouting-checkpoint, max_size=512000

However I'm not sure my parsing queue is big enough if I rely on the largest_size > max_size_kb:

02-08-2017 20:59:26.082 +0100 INFO Metrics - group=queue, ingest_pipe=1, name=parsingqueue, max_size_kb=6144, current_size_kb=0, current_size=0, largest_size=7494, smallest_size=0

I don't have any alert from the Distributed Management Console (DMC), CPU/MEM are fine, anything else I should look at?
Could it be the third party syslog that is not handling all the traffic and cannot ack every packet my HF transmits?

marlongarcia
New Member

Is this issue still outstanding? We are having the same issue. Any possible solution? thanks

0 Karma

sassens1
Path Finder

So till now my research indicates that it may be related to the tcp session timeout on the firewall in between my HF and the remote syslog. to be continued...

0 Karma

dflodstrom
Builder

did this end up being the cause of your issues?

0 Karma

vsingla1
Communicator

@sassens1 Did you find out the root cause that caused this issue?

0 Karma

sassens1
Path Finder

no one? 😕

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...