Getting Data In

How to resolve "TcpOutputProc - Queue for group ICSRouting-checkpoint has begun dropping events" error?

sassens1
Path Finder

Hello,

We use a Heavy Forwarder (HF) to forward CheckPoint logs to an external third-party SIEM using the TCP protocol.
I have noticed from time to time this kind of errors:

01-25-2017 15:47:44.071 +0100 INFO TcpOutputProc - Queue for group ICSRouting-checkpoint has stopped dropping events
01-25-2017 15:47:44.688 +0100 WARN TcpOutputProc - Queue for group ICSRouting-checkpoint has begun dropping events

just a few milliseconds of failure?
I checked my queue size which seems ok:

02-08-2017 20:57:22.077 +0100 INFO Metrics - group=queue, ingest_pipe=0, name=tcpout_icsrouting-checkpoint, max_size=512000

However I'm not sure my parsing queue is big enough if I rely on the largest_size > max_size_kb:

02-08-2017 20:59:26.082 +0100 INFO Metrics - group=queue, ingest_pipe=1, name=parsingqueue, max_size_kb=6144, current_size_kb=0, current_size=0, largest_size=7494, smallest_size=0

I don't have any alert from the Distributed Management Console (DMC), CPU/MEM are fine, anything else I should look at?
Could it be the third party syslog that is not handling all the traffic and cannot ack every packet my HF transmits?

marlongarcia
New Member

Is this issue still outstanding? We are having the same issue. Any possible solution? thanks

0 Karma

sassens1
Path Finder

So till now my research indicates that it may be related to the tcp session timeout on the firewall in between my HF and the remote syslog. to be continued...

0 Karma

dflodstrom
Builder

did this end up being the cause of your issues?

0 Karma

vsingla1
Communicator

@sassens1 Did you find out the root cause that caused this issue?

0 Karma

sassens1
Path Finder

no one? 😕

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...