We have Monitoring of Java Virtual Machines with JMX setup on our Splunk forwarder (linux), and it's running fine when executed "./splunk start" from splunk forwarder bin with below logs.
08-29-2022 09:33:57.733 -0600 INFO SpecFiles - Found external scheme definition for stanza="jmx://" from spec file="/opt/splunkforwarder/etc/apps/SPLUNK4JMX/README/inputs.conf.spec" with parameters="activation_key, config_file, config_file_dir, polling_frequency, additional_jvm_propertys, output_type, hec_port, hec_host, hec_endpoint, hec_poolsize, hec_token, hec_https, hec_batch_mode, hec_max_batch_size_bytes, hec_max_batch_size_events, hec_max_inactive_time_before_batch_flush, log_level"
However, when I tried to start Splunk agent as a service with sudo service splunk start, everything else started fine, and I'm getting the following error in splunkd.log
08-29-2022 09:46:16.519 -0600 ERROR ModularInputs - Introspecting scheme=jmx: Unable to run "python3.7 /opt/splunkforwarder/etc/apps/SPLUNK4JMX/bin/jmx.py --scheme": child failed to start: No such file or directory 08-29-2022 09:46:16.542 -0600 ERROR ModularInputs - Unable to initialize modular input "jmx" defined in the app "SPLUNK4JMX": Introspecting scheme=jmx: Unable to run "python3.7 /opt/splunkforwarder/etc/apps/SPLUNK4JMX/bin/jmx.py --scheme": child failed to start: No such file or directory.
Anyone can point me in the right direction? I've setup Splunk as a service with sudo ./splunk enable boot-start -user splunkuser
I'm suspecting there is a mismatch in permission between splunkuser (splunk owner) and root, but not sure where I should go to correct that.
... View more