I got to the bottom of my problem.
I'm running SPLUNK4JMX 2.2, Splunk 6.2.2 and Java 1.8.0_40
Here's the details:
1) Missing JAVAHOME and SPLUNKHOME variables.
2) Test JMX input by running the following from the command line (this should give you a better clue as to what's happening):
$SPLUNK_HOME/bin/splunk cmd splunkd print-modinput-config jmx | $SPLUNK_HOME/bin/splunk cmd $SPLUNK_HOME/bin/python /opt/app/splunk/splunk/etc/apps/SPLUNK4JMX/bin/jmx.py
3) Check that jmx.py has spawned the java monitoring demon:
14073 ? Ss 0:00 \_ python /opt/app/splunk/splunk/etc/apps/SPLUNK4JMX/bin/jmx.py
14080 ? Sl 8:36 | \_ /opt/app/splunk/jdk1.8.0_40/jre/bin/java -Xbootclasspath/a: -classpath....
4) You may need to remove the SPLUNK4JMX.pid file and restart Splunk:
5) I found that incorrect config caused the java daemon to crash leaving the pid file in place. Double check your config.xml and repeat step 5.
Some other users have had problems with SSLv3 since it was disabled in Java. If you get connection problems under step 2 this may be worth investigating and make sure you are using TLS in jmx.py and web.conf or re-enable SSLv3 in java (not recommended - POODLE) and switch to ssl in jmy.py
Hope this helps
... View more