All Apps and Add-ons

Splunk Stream - Cisco IpFIX . Can't figure out how to ingest IPFIX data for application URL/URI/source/dest

sudoritz
Explorer

Im trying to find some documentation to help aid in ingesting Custom IPfix outside 1-400 IDs. But I read that there's not much documentation in this arena hehe .. Here's what I have tried:

Main goal is to pretty much ingest IPFIX data for application URL/URI / source/dest other Netflow stats but it seems i need to code either in vocabularies or something else.

Cisco ASR 1004 --> streamfwd standalone app --> SH / indexer load

I've noticed Template ID of 294 and enterprise ID of 9

but i dont see it in ipfix.xml in the IETF org assignments

connection client ipv4 address ID = 12236
connection server ipv4 address ID = 12237

i tried setting this in streamfwd.conf

cat streamfwd.conf

[streamfwd]
    port = 8889
    netflowReceiver.0.ip = 10.1.1.1
    netflowReceiver.0.protocol = udp
    netflowReceiver.0.port = 9991
    netflowReceiver.0.decoder = netflow
    netflowReceiver.0.decodingThreads = 8

    netflowElement.0.enterpriseid = 9
    netflowElement.0.id = 12235
    netflowElement.0.termid = cisco.12235
    netflowElement.1.enterpriseid = 9
    netflowElement.1.id = 12236
    netflowElement.1.termid = cisco.12236

and tried setting this in vocabularies

vocabularies]# cat cisco.xml

<?xml version="1.0" encoding="UTF-8"?>
<CmConfig xmlns="http://purl.org/cloudmeter/config" version="7.1.1">
  <Vocabulary id="cisco">
    <Locked>true</Locked>
    <Name>Cisco Netflow Protocol Vocabulary</Name>
     <Term id="cisco.12235">
        <Type>blob</Type>
        <Comment>12235 status.</Comment>
      </Term>
<Term id="cisco.12236">
      <Type>blob</Type>
      <Comment>12236 status</Comment>
</Term>
   </Vocabulary>
</CmConfig>

theres some things im trying to figure out and stitch together like how do i know how to state its a uint32/64

i tried to look at the exporter part of the router to then build in the vocabularies
size1=unsigned8
size4=unsigned32
size8=unsigned64
size32=string
size40=string

but its not 1-1 on some of them so im kinda lost on how i can bridge some of these inbound.

This is the Exporter information from our cisco router showing Template ID of 294 along with IDs and Ent ID

Client: Flow Monitor cisco-flow
  Exporter Format: IPFIX (Version 10)
  Template ID    : 294
  Source ID      : 1280
  Record Size    : 95 + var
  Template layout
  _____________________________________________________________________________
  |                 Field                   |    ID | Ent.ID | Offset |  Size |
  -----------------------------------------------------------------------------
  | connection client ipv4 address          | 12236 |      9 |      0 |     4 |
  | connection server ipv4 address          | 12237 |      9 |      4 |     4 |
  | ip dscp                                 |   195 |        |      8 |     1 |
  | ip protocol                             |     4 |        |      9 |     1 |
  | connection client transport port        | 12240 |      9 |     10 |     2 |
  | connection server transport port        | 12241 |      9 |     12 |     2 |
  | routing vrf input                       |   234 |        |     14 |     4 |
  | connection initiator                    |   239 |        |     18 |     1 |
  | connection id                           | 12242 |      9 |     19 |     4 |
  | flow observation point                  |   138 |        |     23 |     8 |
  | application id                          |    95 |        |     31 |     4 |
  | flow direction                          |    61 |        |     35 |     1 |
  | flow sampler                            |    48 |        |     36 |     1 |
  | services waas segment                   |  9252 |      9 |     37 |     1 |
  | services waas passthrough-reason        |  9253 |      9 |     38 |     1 |
  | application http uri statistics         |  9357 |      9 |     39 |   var |
  | application http host                   | 12235 |      9 |     41 |   var |

*** i have it coming in Splunk cause i edited the app /streams/netflow and i see 12235 but it doesnt show like its correct) mabye because i did it with blob.***

Does anyone have example custom Cisco setup (i thought this would be like an easy 1-1

also in my streamfwd log i have this but not sure if i built it right

2017-10-07 11:06:29 WARN 140323433604864 stream.NetflowReceiver - NetFlowDecoder::decodeFlow Unable to decode flow set data. No template with id 261 received for observation domain id 6 from device 10.0.1.1 . Dropping flow data set of size 1358

0 Karma

sudoritz
Explorer

oops i need to move my answer to my comment when approved

0 Karma

sudoritz
Explorer

question1

how do i know from ciscos SIZE output based on the field how to map out

size1=uint8
size2=uint16
size4=uint32
size8= uint64
size32=string
size40=string

The Type tag must consist of one of the following supported data types:
uint8, uint16, uint32, uint64, shortstring, string, longstring, blob.

But not sure how to get size32/40 on shortstring/longstring/blob

i cant find in Cisco where it states on how to on those.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Hey @sudoritz!!!

Read here: https://www.aplura.com/splunk/extending-splunk-stream-vocabularies-using-ipfix/

If you have further questions, comment below, or hop on slack, find the #stream channel, and ping me!!!

Thanks!

lfedak_splunk
Splunk Employee
Splunk Employee

Here's the link to Slack! http://splk.it/slack

0 Karma
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...