Splunk Search

Create charts from SNMP Counter data type

rasingh
Path Finder

I am collecting snmpget data from a SAN switch. A few of the SNMP elements use counters where I get the accumalated value on each SNMPget. We want to graph the value of the difference between the last value and the current value of that counter over time. I think a new field would need to be created to show this difference before graphing.

This would be similar to say graphing port throughput in Cacti/MRTG/RRDtool.

2 Solutions

gkanapathy
Splunk Employee
Splunk Employee

lukeh
Contributor

We managed to get it working 🙂

We are using this splunk search with Network Traffic statistics sourced from Nagios Plugin Performance Data :-

index=nagios plugin="Network Traffic" hostname="server01"
| sort _time
| delta inAbsolut as inabs_change
| delta outAbsolut as outabs_change
| where inabs_change>=0
| where outabs_change>=0
| eval inkilobits=(inabs_change*8/1000)
| eval outkilobits=(outabs_change*8/1000)
| timechart span=5m per_second(inkilobits) as in_kbps, per_second(outkilobits) as out_kbps

Remember to select Null values: 'Connect' when displaying the above Powerful Graph after clicking "Show report" 🙂

Note: you need to include the 'where' statements to exclude negative delta values (caused by counter resets)

I will be including the above Powerful Graph when I release Splunk for Nagios v. 1.1 in Oct '10, as well as powerful graphs of other metal level metrics (cpu, memory, swap, load, disk usage, processes, etc) sourced from Nagios Plugin Performance Data 🙂

View solution in original post

lukeh
Contributor

We managed to get it working 🙂

We are using this splunk search with Network Traffic statistics sourced from Nagios Plugin Performance Data :-

index=nagios plugin="Network Traffic" hostname="server01"
| sort _time
| delta inAbsolut as inabs_change
| delta outAbsolut as outabs_change
| where inabs_change>=0
| where outabs_change>=0
| eval inkilobits=(inabs_change*8/1000)
| eval outkilobits=(outabs_change*8/1000)
| timechart span=5m per_second(inkilobits) as in_kbps, per_second(outkilobits) as out_kbps

Remember to select Null values: 'Connect' when displaying the above Powerful Graph after clicking "Show report" 🙂

Note: you need to include the 'where' statements to exclude negative delta values (caused by counter resets)

I will be including the above Powerful Graph when I release Splunk for Nagios v. 1.1 in Oct '10, as well as powerful graphs of other metal level metrics (cpu, memory, swap, load, disk usage, processes, etc) sourced from Nagios Plugin Performance Data 🙂

rasingh
Path Finder

I just tested and noticed a couple of things:
1. The 'where' statements should replace '>0' with '>=0' to remove negative delta values.
2. In comparing our Cacti/RRDtool graphs with splunk graphs, we selected "Connect" instead of "Treat as zero" on the graph to make them the same.

0 Karma

rasingh
Path Finder

I have tried generating the graphs but they do not seem accurate. What concerns me right now is that the counters do not operate as I think they do. When I do delta on TXwords say, I get negative numbers. That whould only happen if counter value resets.

0 Karma

lukeh
Contributor

I am also looking to do the same thing, could you please share the search that you used to generate the requisite powerful graph? 😛

0 Karma

rasingh
Path Finder

First, I installed net-snmp and the switch MIBs (Brocade in this case). I use the 'snmptable' command to query all the ports at once and output into csv format - there are several command line options for that.

Once the command polling is tested, I created a batch file in $SPLUNK_HOME/bin as a wrapper to the snmptable command in the install location.

Lastly, I added the batch file with parameters as a scripted input.

Dan
Splunk Employee
Splunk Employee

Can you share how you are doing the snmp polling and putting the events into Splunk?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...