How to calculate Bandwidth Utilization Using SNMP?

[dailv1808]

This is snmp log of some network device interface. How to calculate Bandwidth each interface?

IF-MIB::ifName."436207616" = "Ethernet1/1" IF-MIB::ifOutOctets."436207616" = "2206740419" IF-MIB::ifInOctets."436207616" = "1813149511" IF-MIB::ifSpeed."436207616" = "4294967295" IF-MIB::ifName."436211712" = "Ethernet1/2" IF-MIB::ifOutOctets."436211712" = "1088361704" IF-MIB::ifInOctets."436211712" = "4096669700" IF-MIB::ifSpeed."436211712" = "4294967295" .....

Any reply is welcome, thank for advance!

[FrankVl · Accepted solution]

Ok, been playing around a bit with it myself in the meantime, to grab bandwidth statistics from my asus router.

Few notes on using the SNMP add on:

• it allows you to split the data by setting split_bulk_output = 1 in inputs.conf. Which removes the need for messing with that LINE_BREAKER stuff manually.
• applying sourcetype = snmp_ta get's me some nice field extractions out of the box.

This is the snmp input config I have:

[snmp://asus-wrt]
activation_key = ***
communitystring = public
destination = 192.168.2.1
do_bulk_get = 0
do_get_subtree = 1
host = rt-ac87u-1d50
index = testdata
ipv6 = 0
mib_names = IF-MIB
object_names = 1.3.6.1.2.1.2
port = 161
snmp_mode = attributes
snmp_version = 2C
sourcetype = snmp_ta
split_bulk_output = 1
trap_rdns = 0
v3_authProtocol = usmHMACMD5AuthProtocol
v3_privProtocol = usmDESPrivProtocol

This is the search I came up with to calculate bandwidth usage (example for ifInOctets, note: snmp_index selects 1 specific network interface):

index=testdata sourcetype=snmp_ta snmp_index=8 oid=ifInOctets
| sort _time
| delta ifInOctets as deltaO
| delta _time as deltaT
| table _time,ifInOctets,deltaT,deltaO
| eval KBps = round(deltaO/deltaT/1000,2)

It uses the delta command to calculate the difference in Octet count as well as the difference in time (in seconds) between events and divides the two to calculate the avg KB per second during that timeslot. The | sort _time is needed to get positive delta values and have an actual delta value for your latest event.

An alternative using streamstats (which allows the use of a by-clause)

index=testdata sourcetype=snmp_ta
| eventstats values(ifDescr) as ifDescr by snmp_index
| search oid=ifInOctets
| sort _time
| streamstats window=2 global=false range(ifInOctets) as InOctets range(_time) as secs by ifDescr
| timechart avg(eval(InOctets/secs)) by ifDescr

I'm getting some weird data out of this, which doesn't match the actual bandwidth usage, but I think that may be some issue with the asus device (nat acceleration feature seems to give unreliable network stats), not with how I calculate things in Splunk. Please give it a try and see what you're results are!

[FrankVl]

@dailv1808 did my suggestions get you any closer to solving this?

[dailv1808]

Yes. Thank you so much!

[Richfez]

To add on to FrankVI's comment and to get everyone the information they need to help build you an answer -

Is this data already extracted into fields and values in Splunk, or is that not done yet? (E.g. some variation of
a verbose-modeindex=blah search over some reasonable time period returns fields and values, not just raw logs.)

[dailv1808]

i extracted field using this command on search
| rex max_match=0 "IF-MIB::ifName.\"\d+\"\s=\s\"(?\S+)\""
| rex max_match=0 "IF-MIB::ifInOctets.\"\d+\"\s=\s\"(?\w+)\""
| rex max_match=0 "IF-MIB::ifOutOctets.\"\d+\"\s=\s\"(?\w+)\""
| rex max_match=0 "IF-MIB::ifSpeed.\"\d+\"\s=\s\"(?\w+)\""
| table _time, ifName, ifOutOctets, ifInOctets, ifSpeed

[FrankVl]

On a high level, it is a matter of comparing the difference in Octets values and timestamps from 2 events to calculate how many bytes were sent in x seconds, which can then be translated to (G/M/k)b/s.

How to best do that in Splunk is something I'm not 100% sure on. I'm thinking the delta command might come in handy for that?

[dailv1808]

Thank a lot for your reply. I understood your idea, but there are many interface in each event. So how can i calculate bandwidth on each interface on 1 event? OR how to split 1 event into multi event with just one interface ? sorry my english really not good. plz see my screenshot to understand my meanning. !

[dailv1808]

my screenshot image https://imgur.com/a/rsRA7fb

[FrankVl]

Can you share a screenshot / sample of a raw event? I think you'd really want to split separate interfaces already at index time if possible.

The way you have the data in that screenshot you just shared is already quite useless, because the link between interface and octet values is completely lost once you dump it into multi valued fields (no guarantee that things on the same line in your screenshot actually belong together).

Edit: oh, wait, that was already shared in your question. You could try configure a LINE_BREAKER on IF-MIB::ifName, such that each interface get's ingested as a separate event. Although that might give some headaches with timestamping...

Alternatively, you could at search time, use rex to extract the whole entry for each interface into a field called "ifEntry" or something. Then use mvexpand to split that into separate events. And only then extract the individual fields like ifname and octects.

[dailv1808]

i used snmp input to collect. screenshot of raw event at link below.
https://imgur.com/a/4AJLy0m

I think need to split event to multi event first

[FrankVl]

See the edit to my comment for some ideas on that.

[dailv1808]

nice idea, i tried but it didn't work

https://imgur.com/a/Fu3hGbv

[FrankVl]

Can you press the "copy to clipboard" button in that screen, to see what it actually has configured? I had some trouble getting splunk to properly take LINE_BREAKER and not keep doing some "break only before" setting instead.

Manually defining a sourcetype as follows in props.conf works like a charm:

[snmptest]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
LINE_BREAKER=(\s+)IF-MIB::ifName

Anyway, even if you cannot get line breaking to work, you could still look at my other suggestion, to do the rex field extractions in search time in 2 steps: first extract entire entries with ifname and octet values, then apply mvexpand to split into separate events, then apply rex similar to what you had already to extract individual fields.

[dailv1808]

Many thank FrankVI
I tried again but it still not work i dont know why. You can check my screenshot image link below:
https://imgur.com/a/3ByhMTn

[FrankVl]

Like I said: define those settings in a props.conf file. E.g. I put the following into props.conf under etc/apps/search/local/:

[snmptest]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
LINE_BREAKER=(\s+)IF-MIB::ifName
category = Custom
description = snmptest
disabled = false
pulldown_type = true

And then select that sourcetype in the data import wizard, rather than using that dialog to configure certain settings. The dialog behaves weirdly (I guess because it doesn't work well with data that doesn't actually contain any newlines).

https://imgur.com/PWrMUxz

[dailv1808]

yes, thank you. LINE_BREAKER did work fine when created props.conf file for sourcetype. Now timestamp is incorrect, how can i config to correct timestamp?

[FrankVl]

Perhaps remove that "DATETIME_CONFIG=CURRENT", not entirely sure why I included that.

[dailv1808]

It did work, thank you so much.
Now back to the original problem, do you know how to calculate bandwidth for each interface.

I refer to this link: https://www.cisco.com/c/en/us/support/docs/ip/simple-network-management-protocol-snmp/8141-calculate...

How to calculate the formula in splunk?