All Apps and Add-ons

Splunk Stream - Netflow v9 - Certain fields not really show up

jvanhambelgium
Engager

Hi,

I've got the latest Splunk Stream app installed & configured to accept Netflow v9 events from my router. This part works fine actually.
However, when starting to dig deeper in "useful" fields it seems I'm missing a few...I would expect the Stream App to be able to cope with everything "standard" within v9/IPFIX packets/templates.

When going to the STM App -> Configuration -> Configure Stream -> Netflow -> "Edit" you get this nice list of about 156 fields which I all enabled.

 

Now I've taken Wireshark capture from the v9 data arriving at my Splunk server and the "template" contains these fields below.

Field (20/23): postNATSourceIPv4Address
Type: postNATSourceIPv4Address (225)
Length: 4

Field (21/23): postNATDestinationIPv4Address
Type: postNATDestinationIPv4Address (226)
Length: 4

Field (22/23): postNAPTSourceTransportPort
Type: postNAPTSourceTransportPort (227)
Length: 2

Field (23/23): postNAPTDestinationTransportPort
Type: postNAPTDestinationTransportPort (228)
Length: 2

 

And a typical populated capture would look like this :

Cisco NetFlow/IPFIX
Version: 9
Count: 7
SysUptime: 873590.040000000 seconds
Timestamp: Jun 22, 2020 10:03:08.000000000 CEST
CurrentSecs: 1592812988
FlowSequence: 22
SourceId: 0
FlowSet 1 [id=256] (7 flows)
FlowSet Id: (Data) (256)
FlowSet Length: 532
[Template Frame: 1]
Flow 1
[Duration: 0.000000000 seconds (switched)]
StartTime: 873528.130000000 seconds
EndTime: 873528.130000000 seconds
Packets: 1
Octets: 86
InputInt: 15
OutputInt: 14
SrcAddr: IP.OF.INTERNAL.PC
DstAddr: SOME.PUBLIC.ISP.DNSADDRESS
Protocol: UDP (17)
IP ToS: 0x00
SrcPort: 51020 (51020)
DstPort: 53 (53)
NextHop: SOME.PUBLIC.ISP.DNSADDRESS
DstMask: 0
SrcMask: 0
TCP Flags: 0x00
Destination Mac Address: Router12_12:12:c6 (61:3c:61:31:11:b1)
Source Mac Address: ASRockIn_84:01:36 (d0:50:99:84:01:36)
Post Source Mac Address: 00:00:00_00:00:00 (00:00:00:00:00:00)
Post NAT Source IPv4 Address: MY.PUBLIC.ISP.ADDRESS
Post NAT Destination IPv4 Address: SOME.PUBLIC.ISP.DNSADDRESS
Post NAPT Source Transport Port: 51020
Post NAPT Destination Transport Port: 53

 

Looking again at my Splunk Enterprise intstallation, there is this "vocabulaire" file under

opt/splunk/etc/apps/Splunk_TA_stream/default/vocabularies

It contains the "netflow.xml" file with (what I think) all field that can be "interpreted" / "decoded" as the Netflow packets arrive.

<Term id="netflow.postNATSourceIPAddress">
<Term id="netflow.postNATDestinationIPAddress">
<Term id="netflow.postNAPTSourceTransportPort">
<Term id="netflow.postNAPTDestinationTransportPort">

So ... these 4 fields seem to already be part of the default vocabulary ... yet they never show up as any accessible "field" in Splunk ?

In a moment of madness, I've edited the file below and make some additions.

(router = Mikrotik = IANA Vendor ID 14988)

user@splunky:/opt/splunk/etc/apps/Splunk_TA_stream/default# more streamfwd.conf
[streamfwd]
port = 8889
ipAddr = 127.0.0.1

netflowReceiver.0.ip = IP.OF.MY.SPLUNK
netflowReceiver.0.port = 9995
netflowReceiver.0.decoder = netflow

netflowElement.0.enterpriseid = 14988
netflowElement.0.id = 225
netflowElement.0.termid = netflow.postNATSourceIPAddress

netflowElement.1.enterpriseid = 14988
netflowElement.1.id = 226
netflowElement.1.termid = netflow.postNATDestinationIPAddress

netflowElement.2.enterpriseid = 14988
netflowElement.2.id = 227
netflowElement.2.termid = netflow.postNAPTSourceTransportPort

netflowElement.3.enterpriseid = 14988
netflowElement.3.id = 228
netflowElement.3.termid = netflow.postNAPTDestinationTransportPort

 

...then stop/start my Splunk but these fields don't show up with the 156 possible fields under the "stream config" tab (see earlier)

To cut a long story short : Where are these fields ? Why are they not showing up since they are hitting the Splunk Stream App and it seems they are "known"

Thanks!

Labels (2)

diptij
Loves-to-Learn Everything

Did you figure this out?  if so, what did you do?

0 Karma

jvanhambelgium
Engager

Hi,

Actually never managed to get it to work.

I must also admit the v9/IPFIX Netflow on the Mikrotik RouterOS is not without its problems either...

So for the moment I've settled for Netflow v5 which at least gives me some output. This works fine.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!