All Apps and Add-ons

Ingesting a PCAP of netflow data from a monitored directory.

apollak
Explorer

Hello Spelunkers,
I have a PCAP file that contains only netflow data and a directory that is being monitored for PCAPS by Splunk Stream. I would like to be able to ingest this PCAP into Splunk by dropping it into the monitored directory (which is on the same machine as my Splunk instance). I have been able to ingest normal PCAPs without problem from this directory.

I expected to just be able to put this special netflow PCAP into this directory and have the netflow records be searchable. Instead, two events gets indexed, one from source = stream:Splunk_Udp and the other from source = stream:Splunk_IP. The UDP one says app:netflow but has no data that you would associate with netflow while the IP one has netflow information. From what I can tell, stream is treating all of the netflow information we are sending to it as one flow (this makes sense as it is all to and from the same IP). Is there any way to 'dissect' the big flow into the individual netflow records? If I open the PCAP in wireshark I can see each individual netflow record.

Thanks! Asher

0 Karma
1 Solution

apollak
Explorer

So I pretty much solved it, recording the answer here for posterity.

Ended up using tcprewrite to change the source IP, Dest IP, Destination MAC address and ports -

tcprewrite \
--infile original_pcap.pcap \
--outfile newpcap.pcap \
--srcipmap oldsrcIP:newsrcIP \
--dstipmap olddestIP:newdestIP(splunk) \
--enet-dmac newdestMAC(network adapter splunk is collecting netflow on) \
--portmap=oldport:newport(port splunk is listening on) \
--fixcsum

Then I used tcpreplay to replay the pcap back out to the network. This makes it look like the srcip address is generating the netflow records. Stream had no problem ingesting the netflow in this way. The only problem was the original timestamps were not preserved. This I do not think there is a way to fix as tcpreplay strips the header and replaces the timestamp with its own.

View solution in original post

0 Karma

apollak
Explorer

So I pretty much solved it, recording the answer here for posterity.

Ended up using tcprewrite to change the source IP, Dest IP, Destination MAC address and ports -

tcprewrite \
--infile original_pcap.pcap \
--outfile newpcap.pcap \
--srcipmap oldsrcIP:newsrcIP \
--dstipmap olddestIP:newdestIP(splunk) \
--enet-dmac newdestMAC(network adapter splunk is collecting netflow on) \
--portmap=oldport:newport(port splunk is listening on) \
--fixcsum

Then I used tcpreplay to replay the pcap back out to the network. This makes it look like the srcip address is generating the netflow records. Stream had no problem ingesting the netflow in this way. The only problem was the original timestamps were not preserved. This I do not think there is a way to fix as tcpreplay strips the header and replaces the timestamp with its own.

0 Karma

wenthold
Communicator

If you want to preserve the timestamps you could use something like tcpdump to dump the pcap to a intermediate text file, then read that and parse it all as network traffic.

0 Karma

apollak
Explorer

How do I get that data into Splunk then?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...