All Apps and Add-ons
Highlighted

SNMP traps not being indexed by SNMP Modular Input

Engager

The env:

Win2k8r2

Splunk 6.1 Ent

SNMP Modular Input 1.2.4

The problem:

No traps are being indexed

The stanza

[snmp://iDRAC]
communitystring = public
do_bulk_get = 0
index = myindex
ipv6 = 0
snmp_mode = traps
snmp_version = 1
sourcetype = snmp_traps
split_bulk_output = 0
listen_traps = 1
trap_host = localhost
trap_port = 162
v3_authProtocol = usmHMACMD5AuthProtocol
v3_privProtocol = usmDESPrivProtocol

Note: the stanza was generated by splunk web. The dynamically generated entry did NOT contain "listen_traps = 1" which appeared to be needed according to the readme. So I added that line manually. The behavior was unchanged by this.

Steps taken:

  • I installed wireshark and verified that the trap was indeed making it to the box splunk runs on, and that it was coming on the correct port.
  • I tested with windows firewall enabled and disabled, which made no difference
  • I have a rule in windows firewall explicitly allowing UDP 162 for snmp traps
  • I ran netstat and verified that the host is listening on UDP 162

So, I can show the the trap is received by the host. I don't believe the firewall is interfering and testing with it disabled yielded the same results. I believe that leave either the snmp daemon that snmp_ta is using, some other misconfiguration on my part or something else on the host causing some sort of conflict.

Splunk is the only real utility running on the host, as that is its sole purpose. But I'm not 100% if there is anything native to Windows that could interfere with snmp handling.

Has anyone hit this? Any suggestions? Is there any other info that would be helpful?

Tags (1)
0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Motivator

do you have anything in splunkd.log ? (index=_internal sourcetype=splunkd snmp)

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Engager

index=internal sourcetype=snmptraps yields nothing in the logs. And just looking at the recent _internal doesn't show any errors minus one unrelated to this data input.

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Ultra Champion

Firstly , you do not need to manually add "listentraps = 1" , that is just a legacy property for pre 1.2.2 support.The "snmpmode" property is the new way forward.

What do you see if you search over "all time" ?

Any errors in "index=_internal ExecProcessor error snmp.py" ?

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Engager

Ok thanks. I'll take that back out of the stanza.

All searches listed will be over "all time". The splunk installation isn't very old, so there isn't much to search. Running the search you named above yielded one result... but it sort of looks like the search generated it.

I'll post another reply with the contents, as the message puts me over the char limit.

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Engager

The entry:
[05/Jun/2014:08:52:24.986 -0400] "GET /en-US/api/shelper?snippet=true&snippetEmbedJS=false&namespace=search&search=search+index%3Dinternal+ExecProcessor+error+snmp.py&useTypeahead=true&useAssistant=true&showCommandHelp=true&showCommandHistory=true&showFieldInfo=false&=1401968578523 HTTP/1.1" 200 579 "http://:8000/en-US/app/search/search?q=search%20index%3Dinternal%20sourcetype%3Dsnmptraps&sid=1401968706.126&earliest=&latest=" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36" - 53906808fc437fe10 168ms

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Engager

Well, I have no doubt that this app works for most people. But it sure doesn't work for me and with sparse documentation and "community support" there isn't much hope for getting it to actually do anything.

I guess I'll try to rig net-snmp. It seems super clunky compared to a built in solution... but it works.

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Path Finder

I'm having the same issue. No errors either. I double checked that I'm getting data on UDP port 162 by listening on that port with Netcat. Were you able to solve this?

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Ultra Champion

Can you email me a wireshark capture of the traps that are not showing up in Splunk ? ddallimore@splunk.com

0 Karma
Highlighted

Re: SNMP traps not being indexed by SNMP Modular Input

Communicator

I had this same problem and fixed it by changing traphost to match what was set up in the device that was sending the snmp traps. In my case we used a DNS entry to point the traps to the Splunk Indexer/SH and when I entered that DNS entry in for traphost instead of "localhost" then the SNMP traps immediately started showing up (I almost positive that this requires a restart)

I actually found this post while looking up another question about this app that maybe you can answer, can this app be used on a forwarder instead of directly on the indexer? I'm assuming it can since its a TA but I wasn't for sure.

View solution in original post