I am trying to reproduce a case where the following error occurs in splunkd.log and as a result data collection stops:
09-24-2019 21:44:45.713 +0200 ERROR ExecProcessor - message from "python /$SPLUNK_HOME/etc/apps/Splunk_TA_jmx/bin/jmx.py" Exception in thread "Thread-20" java.lang.OutOfMemoryError: Java heap space
In this case, the datacollection resumes only after restarting Splunk.
I am able to get OutOfMemory Error by reducing the maximum heap space in java by making the following changes in file
$SPLUNK_HOME/etc/apps/Splunk_TA_jmx/bin/java_consts.py (line 43)
JAVA_COMMON_ARGS = [JAVA_EXECUTABLE, "-Xmx8m","-classpath", CLASSPATH, "-Dconfighome=" + CONFIG_HOME,
"-Dsplunkhome=" + SPLUNK_HOME]
However, in my case, the OutOfMemory Error goes away after a few minutes and datacollection resumes.
For the issue I am trying to reproduce, 151 inputs are configured in the addon.
Can anyone tell why this error occurs, how can I reproduce it and is there a way to resolve it?