How to get traffic trend graph using snmpget information?

[melonman]

Hi,

I am trying to create network traffic trend graph by:

1. execute snmpget to network devices every minute to get counter value for IN/OUT in Octets and put the output to splunk as scripted input.

2. extract counter field to get counter value, convert counter value to decimal, get delta to get bit per minute, then get bps by dividing by 60.

3. use timechart command to get avg(bps) in 5min.

I can see the scripted inputed is successfully indexed into splunk, but I can't get 5-minute average bps using the folloing search:

sourcetype=snmpget host=192.168.1.1 mib_name=*ifIn* 
  | eval decimal=tonumber(counter,8) 
  | delta decimal as bpm 
  | eval bpm = abs(bpm) 
  | eval bps=(bpm/60) 
  | timechart avg(bps) span=5m

the search result looks like this:

           _time             avg(bps)
--------------------------- -----------
2010-10-27 15:30:00.000 CST
2010-10-27 15:35:00.000 CST
2010-10-27 15:40:00.000 CST
2010-10-27 15:45:00.000 CST
2010-10-27 15:50:00.000 CST  837.983333
2010-10-27 15:55:00.000 CST  940.983333
2010-10-27 16:00:00.000 CST  942.333333
2010-10-27 16:05:00.000 CST 1250.377778
2010-10-27 16:10:00.000 CST 3151.966667
2010-10-27 16:15:00.000 CST 3144.083333
2010-10-27 16:20:00.000 CST
2010-10-27 16:25:00.000 CST 1466.822222
2010-10-27 16:30:00.000 CST
2010-10-27 16:35:00.000 CST
2010-10-27 16:40:00.000 CST
[root@syslog1010 ~]#

There are several gaps. Is there any usage mistake in my query? or if anyone know how to achieve this kind of traffic graph, please let me know.

Thanks..

[rasingh · Accepted solution]

Check out the answer here.

http://answers.splunk.com/questions/1198/create-charts-from-snmp-counter-data-type

[gkanapathy · Accepted solution]

Something like this:

sourcetype=routerifsnmpinfo
 | streamstats current=f global=f window=1
     first(ifHCInOctets) as next_ifHCInOctets,
     first(ifInErrors) as next_ifInErrors,
     first(_time) as next_time
   by host,ifIndex
 | eval dt=next_time-_time
 | eval difHCInOctets=next_ifHCInOctets-ifHCInOctets
 | eval rifHCInOctets=difHCInOctets/dt
 | eval cifInErrors=next_ifInErrors-ifInErrors

is more resistant to skips or delays in the timing of collections of data. The above assumes that ifHCInOctets and ifInErrors are counters, and ignores the 32-bit counter issue by using the HC version of the counter, which is supposed to be a 64-bit counter. Note the use of streamstats rather than delta since it allows you to split by host and interface number (ifIndex) and get a full table instead of having to query one host/interface at a time.

[rasingh · Accepted solution]

Check out the answer here.

http://answers.splunk.com/questions/1198/create-charts-from-snmp-counter-data-type

[dwaddle]

I don't see anything obvious about your search that looks out of whack - are you getting the raw data indexed for all of your intervals?

That said, there's some tricksy stuff about SNMP counters. Eventually, they will roll over so you need to be prepared for that. The rollover could be from a true counter rollover (some devices use 32 counters for that kind of thing, and roll over relatively quicky), or from a device reboot, or an SNMP agent restart.

There are many tools "out there" already that handle this problem well - stuff like Cacti, MRTG, and Cricket. I'm wondering if you wouldn't find as much success trying to leverage their existing data collection tools to feed into Splunk for graphing/analysis.