I have an all in one Splunk VM setup with Splunk stream installed. I have a mirror port on my switch that is tapping all LAN traffic going into the router and then to the internet (monitoring everything outbound and inbound on LAN side interface of firewall). I have a dedicated NIC in the VM that is connected via direct IO as my TAP, and different Virtual NIC tied to a different physical nic for management of the VM and the splunk web interface, etc.
I have used tcpdump with the second direct IO NIC in the VM to verify that i am seeing traffic going both ways. I have also pinged an internet host and within splunk i see echo request and reply for the 4 pings. The VM has 8 Vcpu and 64gb of RAM. I also increased the setting for maxTcpReassemblyPacketCount to 1000000.
Still i have 2 problems:
Errors in stream logs. This one i see once: stream.NetworkCapture - decodePacket: Unsupported IP version:
This other error repeats periodically: stream.PacketProcessor - TCP reassembly queue overflow
I seem to be missing most stats for SSL, and when i search for SSL certs (subject name) i get one or two from days ago. The error listed above (TCP assembly queue overflow) almost always has a destination IP address with port 443.
I am assuming things are related. But i can't find any where else to look. Any suggestions?
