Getting SNMP Data into Splunk

[msarro]

Hey everyone, I am trying to figure out the most efficient way to get polled SNMP data into splunk. Strangely while there appears to be a JMX extension for Splunk, there is nothing for SNMP which is far more prevalent. I was considering writing a script to poll numerous servers using net-snmp, and then write to a flat text file, letting splunk sort out the results but that causes some issues with handling table data. Anyone who has tackled this, I would love to hear how you resolved it.

Edit:
Just to provide some information - the SNMP queries we are executing are looking, in some cases, at several hundred different OID's. There is no realistic way to manually script out every single thing I'd like to query. By far the preference would be some sort of app I could drop the MIB into place and go from there.

[cpt12tech]

Here's another solution you can try:
https://answers.splunk.com/answers/521362/found-a-simple-snmp-trap-receiver-for-windows-that.html#an...

[woodcock]

There are now several solutions for this including:
https://splunkbase.splunk.com/app/1537/
http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/SendSNMPeventstoSplunk
http://answers.splunk.com/answers/229041/how-to-send-snmp-traps-from-my-linux-machine-to-a.html

[Damien_Dallimor]

Check out this new add-on : http://splunk-base.splunk.com/apps/88686/snmp-modular-input

[bmacias84]

If you are trying to poll a SNMP agent I have had great success with pysnmp. Check out these examples from pysnmp module. Module comes with a snmpwalker, snmpget, and other utilities.

http://pysnmp.sourceforge.net/examples/4.x/v3arch/manager/bulkgen.html
http://pysnmp.sourceforge.net/examples/4.x/v3arch/manager/getgen.html

Also consider making your script threaded allowing you to poll multiple agents. Though when trying to store responses from the cbFun function and cbCtx var be aware that these are callback objects.

In the end I end up with the following output:

date time object name\instance value
2012-07-13 00:02 cfwBufferStatValue maximum number of allocated 0 byte blocks 1450
2012-07-13 00:02 cfwBufferStatValue fewest 0 byte blocks available 1433
2012-07-13 00:02 ciscoMemoryPoolUsed System memory 821745384
2012-07-13 00:02 ciscoMemoryPoolUsed MEMPOOL_DMA 54508856
2012-07-13 00:02 ifHCInOctets internal 1551153584
2012-07-13 00:02 ifHCInUcastPkts external 13477162449
2012-07-13 02:17 cpmCPUTotal1min 1 3
2012-07-13 02:17 cpmCPUTotal1min 1 3

You will have to convert your mib files to a pysnmp acceptable format to obtain the oid’s name or instance, then importing them using builder in the pysmp module.

Update: Here is a post from me what includes sample code of a proto type I wrote.

snmpget-snmptrap-splunk-and-snmp-polling

[Damien_Dallimor]

Are you trying to poll metrics from a JVM/JVM app , or are you just talking about SNMP MIB polling in general ? I ask because you make mention of Splunk4JMX in your opening post.

I was going to write a blurb on JMX vs SNMP for JVM/JVM app monitoring , but THIS BLOG covers the bases pretty well.

[ChrisG]

There are a couple of other Splunk Answers postings that point to scripted inputs as a solution, too:

http://splunk-base.splunk.com/answers/38219/snmpget-with-splunk

http://splunk-base.splunk.com/answers/38219/snmpget-with-splunk

[Brian_Osburn]

I've actually attempted to do this with our Weblogic instances using a PERL script.

I abandoned it for Splunk4JMX but I still have the PERL script that could modified to do what you need it to do.

SNMP isn't so simple in my mind 🙂

[mschmttco]

snmptrapd will allow snmp traps to be sent to splunk, but polling snmp tables will require a poller. Net-Snmp scripts will work, however I have found integrating Splunk with NerveCenter to be a better more flexible and efficient way to accomplish this. With NerveCenter I can poll the data and format/massage the data easily before sending to splunk. An example to to poll if tables and use NerveCenter to calculate the deltas from the counters and then calculate the % of bandwidth usage to be logged to Splunk. I can also use NerveCenter to apply logic to polled data for alerting. See www.logmatrix.com for more details on NerveCenter.

[ChrisG]

The basics are covered in the documentation topic Send SNMP events to Splunk. It describes how to use snmptrapd to write the data to a file on your Splunk server. What issues are you seeing with the table data?

[cqian02]

I have set Splunk to monitor the snmptrapd.log file, I can see changes in Splunk everytime I restart the Trap Handler Service. So my problem is I can't see any data traffic into my log file, I wonder if I configure snmptrapd.conf wrong.

snmpTrapdAddr [System IP]:162 -- I added the IP where Splunk is installed on
authCommunity log [community string] -- I changed the string to public

Thank you very much.

[ChrisG]

Reviewing the basics, just to narrow down the problem. The steps are:

1. Configure the remote devices to send their traps directly to the Splunk server's IP address.The default port for SNMP traps is udp:162.
2. Write the SNMP traps to a file on the Splunk server using snmptrapd.
3. Configure Splunk to monitor the file,

Are you using monitor and still not seeing any data?

[cqian02]

Hi Chris, after I configured snmptrapd, I just get a file with version information, but no data flows seen so far. Which host shall I configure snmptrapd service on? Thank you.

[msarro]

Sadly I'm less worried about traps, and more about polling which that guide doesn't cover (per the statement near the top). Any other ideas would be really appreciated. It seems strange that something far less implemented like JMX has an app, but not SNMP.