Monitoring of Java Virtual Machines with JMX: How to troubleshoot error "Unable to initialize modular input "jmx""?

[abassili]

I have successfully installed the JMX app (version 2.2) on my Splunk server (version 6.1).

I have tried to modify the "config.xml" file that comes with the app and updated only this part to monitor the "jconsole" app through JMX. I have restarted Splunk, but every time I get the following error:

"Unable to initialize modular input "jmx" defined inside the app "SPLUNK4JMX": Introspecting scheme=jmx: script running failed (exited with code 1)."

I suspect that the "config.xml" file has not been configured properly, but I'm not sure what the problem is.

The splunk log files do not show useful information.

Any help is appreciated.

[icegras · Accepted solution]

We got the same issue and figured out why. JAVA_HOME was pointing to a 1.6 JDK. We changed it to a 1.7 JDK and the error went away.

[chandankp]

Hi I am using PCF JMX Monitor from : https://github.com/mjseid/splunk-pcf-monitor
After all the changes, setting up JAVA_HOME and SPLUNK_HOME and commenting jdk.tls.disabledAlgorithms=SSLv3

I am still getting : Unable to initialize modular input "jmx" defined inside the app "splunk-pcf-monitor-master": Introspecting scheme=jmx: script running failed (exited with code 1)",

[icegras]

This is a whole different plugin, I really cannot say. You should open a new question.

[abassili]

Yes, updating JAVA_HOME solved the problem.

[austenconstable]

Hi,

I got to the bottom of my problem.

I'm running SPLUNK4JMX 2.2, Splunk 6.2.2 and Java 1.8.0_40

Here's the details:

1) Missing JAVA_HOME and SPLUNK_HOME variables.

Edit

 ~/.profile

add

export JAVA_HOME=/opt/app/splunk/jdk1.8.0_40/jre
export SPLUNK_HOME=/opt/app/splunk/splunk

2) Test JMX input by running the following from the command line (this should give you a better clue as to what's happening):

$SPLUNK_HOME/bin/splunk cmd splunkd print-modinput-config jmx | $SPLUNK_HOME/bin/splunk cmd $SPLUNK_HOME/bin/python /opt/app/splunk/splunk/etc/apps/SPLUNK4JMX/bin/jmx.py

3) Check that jmx.py has spawned the java monitoring demon:

ps fax
...
14073 ?        Ss     0:00      \_ python /opt/app/splunk/splunk/etc/apps/SPLUNK4JMX/bin/jmx.py
14080 ?        Sl     8:36      |   \_ /opt/app/splunk/jdk1.8.0_40/jre/bin/java -Xbootclasspath/a: -classpath....

4) You may need to remove the SPLUNK4JMX.pid file and restart Splunk:

rm $SPLUNK_HOME/etc/apps/SPLUNK4JMX/SPLUNK4JMX.pid

5) I found that incorrect config caused the java daemon to crash leaving the pid file in place. Double check your config.xml and repeat step 5.

Some other users have had problems with SSLv3 since it was disabled in Java. If you get connection problems under step 2 this may be worth investigating and make sure you are using TLS in jmx.py and web.conf or re-enable SSLv3 in java (not recommended - POODLE) and switch to ssl in jmy.py

Hope this helps

[internet_team]

Hi,

thanks for tricks, i confirm as sslv3 was disabled, jmx monitoring failed.

To fix temporary, edit java config file in $JAVA_HOME/lib/security/java.security to comment line

jdk.tls.disabledAlgorithms=SSLv3

Now we have to schedule ssl enable on Splunk...

[abassili]

Sadly, no.

[austenconstable]

Hi,

Did you get to the bottom of this?