SNMP Modular Input Poller: What is the best way to log multi-dimensional data in a format that makes it easy to perform stats & timecharts?

[joxley]

I have built the snmpmod modular input for polling network interfaces and Cisco IPSLA statistics. I want to add support for QoS policy shaping. The problem is I don't know how best to represent the data in Splunk.

• Each event (collected every 5 minutes) will be for one interface in one direction (in/out)
• There are multiple QoS class maps such as REALTIME, IN-CONTRACT, etc (6 to 8 of them)
• There are multiple statistics to collect, such as prePolicyBitRate, postPolicyBitRate, prePolicyPkt64, etc

For each class map there is a value for each statistic. The data would look like:

                        REALTIME    IN-CONTRACT   
   prePolicyBitRate         3546          48599   
   postPolicyBitRate      328477         854989   
   prePolicyPkt64            465           9950   

How can I best represent the data in the Splunk event? My ideas so far have been:

<timestamp> interface=1 direction=in REALTIME.prePolicyBitRate=11234 REALTIME.postPolicyBitRate=5433

or

<timestamp> interface=1 direction=in REALTIME="prePolicyBitRate/11234" REALTIME="postPolicyBitRate/5433"

I'm not sure which format would make it easiest to perform stats & timecharts on.

[alacercogitatus · Accepted solution]

I would split it apart even further if possible. For each type of class map, output an event.

 <timestamp> interface=1 direction=in class_map=REALTIME prePolicyBitRate=11234 postPolicyBitRate=5433
 <timestamp> interface=1 direction=in class_map=IN-CONTRACT prePolicyBitRate=11234 postPolicyBitRate=5433

This makes search optimization much easier, as you can specify a class_map= in your search to limit the data you want without having to do complicated extractions. Clean and easy Key Value Pairs are the way to go.

If you can't split like this, then a JSON object approach might work well.

{ "timestamp" : <timestamp>, "direction" : "in", "realtime" : {  "prePolicyBitRate": 11234 }, "in-contract" : { "postPolicyBitRate" : 5433 } }

Or, the poor man's json:

<timestamp> interface=1 direction=in realtime.prepolicybitrate=11234 incontract.postpolicybitrate=5433