I'm having problems getting any SNMP data into Splunk using the SNMP Modular Input. I've set up a Polling Input that is polling a Windows 2008 Server. So far no data has shown up in Splunk.
I've checked for errors using:
index=_internal ExecProcessor error snmp.py
and don't see any results.
communitystring = public
destination = 192.168.1.101
dobulkget = 1
dogetsubtree = 1
host = Archive
ipv6 = 0
snmpmode = attributes
snmpversion = 2C
sourcetype = snmpta
splitbulkoutput = 0
traprdns = 0
v3authProtocol = usmHMACMD5AuthProtocol
v3privProtocol = usmDESPrivProtocol
Are your snmp ports open? Defaults to 161 on the source (called destination in the a snmp app) and 162 on the Splunk server.
If your splunk server is linux and you're following best practices, you will not be able to open port 162 because only root can open ports below 1025.
Thanks for the reply. I'm running Splunk on a Windows server. I'm monitoring the same source with another SNMP management app on a different server and am able to connect and poll the data. The firewall is turned off on the Windows server running Splunk and the Windows server I'm attempting to poll.
Any firewalls between the two on the network?
Use the following command to test:
start -> run -> cmd [enter/click ok]
telnet [ipaddress] [portnumber]
If you get a blinking cursor on a blank page, the port is open to the ipaddress you provided. Else you'll get a timeout error. If you do get the blinking cursor press ctrl + ] to escape out of the telnet session. Try from both servers to the other using the correct ports and ips.
Telnet might not be installed on the server, if not run this at command prompt
Tried the telnet, but there was no response. I searched around the internet and it seems that Telnet doesn't support UDP. I downloaded nmap, and scanned port 161 and it reports as open.
Really great tool.
I can give a couple of things that helped me with the issues I had with SNMP Modular Input. I've found the process pretty cumbersome and required a lot of troubleshooting, at least from Windows.
As an aside, if anyone has issues compiling their MIB files into python, you can actually drop an empty .py file into the custom mib directory, specify that filename in the SNMP Modular Input data input setup, and you will be able to poll. You just don't see the translation of oid values to key name. If the MIB file is specified in your input setup but not present polling will abort for that input.
Does your remote device report any errors like bad community name received or login failure, etc? It seems odd that you are specifying snmp_version = 2C but also including v3 auth settings?
Installed Wireshark, another great tool, and I do see SNMP data coming into the Splunk server, but don't see any SNMP messages leaving the server. I've set up trap notifications on the server's I'm monitoring and assuming those are the SNMP messages being received. Polling should be sending out messages? Searched in the Splunkd log (index=_internal sourcetype=splunkd snmp) and did find some messages.
I set up two trap receivers in Splunk as well. One with localhost as the trap destination, and the other with the IP I'm using to send the SNMP messages to.
03-17-2016 15:44:53.661 -0600 ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\snmpta\bin\snmp.py"" Failed to register transport and run dispatcher: bind() for ('localhost', 162) failed: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions snmpstanza:snmp://Kaleido-Trap
03-17-2016 15:24:32.816 -0600 ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\snmpta\bin\snmp.py"" Failed to register transport and run dispatcher: bind() for (u'192.168.1.102', 162) failed: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions snmpstanza:snmp://Archive-Trap
I've noticed all the trap messages are coming in on port 161. For windows servers I'm using the Microsoft SNMP service. For other devices I have I use whatever they provide. Will the SNMP Modular Input receive from port 161?
I didn't specify an MIB, just wanted to see if I could get any data into the Splunk database. I haven't been able to compile custom MIB's using python and get command not found when trying to make the egg.
How did you find out that the Symantec Endpoint Protection was blocking the UDP port?
When you poll SNMP from splunk, you should see SNMP packets leaving your server get-request and snmp packets returning get-response I believe they show as. I can find a packet capture when I am at office tomorrow. Of particular note, drill into those SNMP packets and verify the UDP port number, etc is as expected.
You are running splunk as a service? Are you running it as LocalSystem or some other user account?
Does your organization push any Microsoft Group Policy, or other firewall/antivirus policy that would block outgoing UDP connections from the machine? Another thing to check would be to try disabling any SNMP service running directly on the server you have splunk installed on, it almost sounds like they may be conflicting?
SNMP Modular Input can listen on any UDP port - specify the port number when you are in the settings page for the Data Input.
This: http://www.alvestrand.no/objectid/220.127.116.11.2.1.1.html is a good set of OIDs to use for test polls. I don't think you have to specify the MIB file name, just pick one or more of those OIDs. They are standard MIBS and I believe included with SNMP Modular Input.
As to SEP - I have an unfortunate long history from previous work with SEP not behaving as it should. I took a chance and repeated my setup on a windows VM that did not have it installed - I put Microsoft Security Essentials on it instead. (use at your own risk). I also had problems with SEP client blocking command line API calls needed to setup splunk clustering.
Your error indicates that port 162 can't be bound to on your Splunk server.
[Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions
My best guess is that you have SNMP Agent or other SNMP service running on the Splunk server and it has already bound to the port. Start -> run -> services.msc [return/click ok]. Look for SNMP services. You can also netstat -ano at command prompt to see if the port is already in use when splunkd is stopped.
To fix, disable whatever is bound to the port.
Other possibility includes splunkd running as underprivileged user (a best practice), and the user needs a Group Policy setting to allow it to bind to 162.
Oh and the easiest solution. Edit the SNMP modular input settings and use TRAP port 1162 or any other port instead of the default 162. or just disable the TRAP by U checking the box