All Apps and Add-ons

All in one lab box - TCP reassembly queue overflow error with stream

andrewm99
New Member

I have an all in one Splunk VM setup with Splunk stream installed. I have a mirror port on my switch that is tapping all LAN traffic going into the router and then to the internet (monitoring everything outbound and inbound on LAN side interface of firewall). I have a dedicated NIC in the VM that is connected via direct IO as my TAP, and different Virtual NIC tied to a different physical nic for management of the VM and the splunk web interface, etc.

I have used tcpdump with the second direct IO NIC in the VM to verify that i am seeing traffic going both ways. I have also pinged an internet host and within splunk i see echo request and reply for the 4 pings. The VM has 8 Vcpu and 64gb of RAM. I also increased the setting for maxTcpReassemblyPacketCount to 1000000.

Still i have 2 problems:

  1. Errors in stream logs. This one i see once: stream.NetworkCapture - decodePacket: Unsupported IP version:
    This other error repeats periodically: stream.PacketProcessor - TCP reassembly queue overflow

  2. I seem to be missing most stats for SSL, and when i search for SSL certs (subject name) i get one or two from days ago. The error listed above (TCP assembly queue overflow) almost always has a destination IP address with port 443.

I am assuming things are related. But i can't find any where else to look. Any suggestions?

0 Karma

jraso
Explorer

Hi andrewm99,

Did you find any solution to this ussue? I have the same problem and I'm loosing a lot of events too!

Thanks!

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...