All Apps and Add-ons

Ingesting a PCAP of netflow data from a monitored directory.

Explorer

Hello Spelunkers,
I have a PCAP file that contains only netflow data and a directory that is being monitored for PCAPS by Splunk Stream. I would like to be able to ingest this PCAP into Splunk by dropping it into the monitored directory (which is on the same machine as my Splunk instance). I have been able to ingest normal PCAPs without problem from this directory.

I expected to just be able to put this special netflow PCAP into this directory and have the netflow records be searchable. Instead, two events gets indexed, one from source = stream:SplunkUdp and the other from source = stream:SplunkIP. The UDP one says app:netflow but has no data that you would associate with netflow while the IP one has netflow information. From what I can tell, stream is treating all of the netflow information we are sending to it as one flow (this makes sense as it is all to and from the same IP). Is there any way to 'dissect' the big flow into the individual netflow records? If I open the PCAP in wireshark I can see each individual netflow record.

Thanks! Asher

0 Karma
1 Solution

Explorer

So I pretty much solved it, recording the answer here for posterity.

Ended up using tcprewrite to change the source IP, Dest IP, Destination MAC address and ports -

tcprewrite \
--infile original_pcap.pcap \
--outfile newpcap.pcap \
--srcipmap oldsrcIP:newsrcIP \
--dstipmap olddestIP:newdestIP(splunk) \
--enet-dmac newdestMAC(network adapter splunk is collecting netflow on) \
--portmap=oldport:newport(port splunk is listening on) \
--fixcsum

Then I used tcpreplay to replay the pcap back out to the network. This makes it look like the srcip address is generating the netflow records. Stream had no problem ingesting the netflow in this way. The only problem was the original timestamps were not preserved. This I do not think there is a way to fix as tcpreplay strips the header and replaces the timestamp with its own.

View solution in original post

0 Karma

Explorer

So I pretty much solved it, recording the answer here for posterity.

Ended up using tcprewrite to change the source IP, Dest IP, Destination MAC address and ports -

tcprewrite \
--infile original_pcap.pcap \
--outfile newpcap.pcap \
--srcipmap oldsrcIP:newsrcIP \
--dstipmap olddestIP:newdestIP(splunk) \
--enet-dmac newdestMAC(network adapter splunk is collecting netflow on) \
--portmap=oldport:newport(port splunk is listening on) \
--fixcsum

Then I used tcpreplay to replay the pcap back out to the network. This makes it look like the srcip address is generating the netflow records. Stream had no problem ingesting the netflow in this way. The only problem was the original timestamps were not preserved. This I do not think there is a way to fix as tcpreplay strips the header and replaces the timestamp with its own.

View solution in original post

0 Karma

Communicator

If you want to preserve the timestamps you could use something like tcpdump to dump the pcap to a intermediate text file, then read that and parse it all as network traffic.

0 Karma

Explorer

How do I get that data into Splunk then?

0 Karma