Ingesting a PCAP of netflow data from a monitored directory.

[apollak]

Hello Spelunkers,
I have a PCAP file that contains only netflow data and a directory that is being monitored for PCAPS by Splunk Stream. I would like to be able to ingest this PCAP into Splunk by dropping it into the monitored directory (which is on the same machine as my Splunk instance). I have been able to ingest normal PCAPs without problem from this directory.

I expected to just be able to put this special netflow PCAP into this directory and have the netflow records be searchable. Instead, two events gets indexed, one from source = stream:Splunk_Udp and the other from source = stream:Splunk_IP. The UDP one says app:netflow but has no data that you would associate with netflow while the IP one has netflow information. From what I can tell, stream is treating all of the netflow information we are sending to it as one flow (this makes sense as it is all to and from the same IP). Is there any way to 'dissect' the big flow into the individual netflow records? If I open the PCAP in wireshark I can see each individual netflow record.

Thanks! Asher

[apollak · Accepted solution]

So I pretty much solved it, recording the answer here for posterity.

Ended up using tcprewrite to change the source IP, Dest IP, Destination MAC address and ports -

tcprewrite \
--infile original_pcap.pcap \
--outfile newpcap.pcap \
--srcipmap oldsrcIP:newsrcIP \
--dstipmap olddestIP:newdestIP(splunk) \
--enet-dmac newdestMAC(network adapter splunk is collecting netflow on) \
--portmap=oldport:newport(port splunk is listening on) \
--fixcsum

Then I used tcpreplay to replay the pcap back out to the network. This makes it look like the srcip address is generating the netflow records. Stream had no problem ingesting the netflow in this way. The only problem was the original timestamps were not preserved. This I do not think there is a way to fix as tcpreplay strips the header and replaces the timestamp with its own.

[wenthold]

If you want to preserve the timestamps you could use something like tcpdump to dump the pcap to a intermediate text file, then read that and parse it all as network traffic.

[apollak]

How do I get that data into Splunk then?