Having trouble getting started with the Java Agent

[wcolgate_splunk]

I would like to instrument my Java application. While I was successful with the monitoring the JVM application (https://splunkbase.splunk.com/app/668/), I'm having a difficult time getting the Java Agent app working.

My environment is Windows 10 Pro, JDK and JRE 1.8.

My java app generates data over HEC to splunk. And I want to splunk my splunk app.

I've followed the instructions in the readme (I removed out the .properties file from the .jar and edited it locally). But running the app with the java agent produces no output, neither by my app, nor by the java agent.

Without the java agent, my app generates data as expected.

I've enabled DEBUG in the agent, and it does read the properties and spits out some data to the log file at what looks like a polling interval; but other than that, nothing else is going on. BTW, the agent is supposed to be target HEC for data, not TCP.

Did any one else have any issues right out of the box? Anything the community learned that might help me debug my situation?

[Damien_Dallimor]

So it looks like you have 2 types of tracing configured to run , method injection tracing and jmx.

1) JMX ..... i see an error loading the jmx.xml config file

12:18:27,459 JMXMBeanPoller ERROR -
The root config object(JMXPoller)
failed to initialize

Did you alter it ?

2) Method Injection Tracing....I see no errors around this , but obviously you'll only get events in Splunk if your JVM program actually does some processing and has threads hit classes and methods you are tracing.You could also try reducing the splunk.transport.internalQueueSize value to flush the buffer sooner under low throughput conditions.

[wcolgate_splunk]

I did not alter the jmx.xml file. Nothing in the instructions indicated I should mess with it for an out-of-the-box installation.

I think that problem #1 is what is holding things back -- because even if #2 were true (i.e. not hitting any of the classes/methods) , my app would be generating output, but does not. The symptom looks like my app doesn't even run at all and hangs.

When using just the monitor the JVM app (https://splunkbase.splunk.com/app/668/), my application runs and generates data, and the monitoring app does poll data from the JVM.

Can your two apps be running at the same time? (i.e. the agent & the JMX monitoring?)

[Damien_Dallimor]

The specific error is the jmx.xml fails to parse for whatever reason in your environment.I only ever see this when the file gets changed and then has a syntax error.Maybe something "windows'y" added weirdness (blank spaces) to the config xml ?Guessing.

Nothing to do with anything else.

You could try turning off the jmx polling , trace.jmx=false

You get exactly the same data from the agent's jmx polling as you do via using the seperate JMX app, it's actually the same underlying code.

[wcolgate_splunk]

So I can drop the JMX polling app (the mod input) once I can get the agent working? Sounds good. The java agent, with the byte code injection/tracing is where I want to end up.

[wcolgate_splunk]

log file: ((changed the extension to satisfy the upload constraints))

link text

properties file: ((changed the extension to satisfy the upload constraints))

link text

CLI:

java.exe -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -javaagent:splunkagent.jar=splunkagent.properties -jar
C:\wrk\git\simdata\build\libs\simdata-1.0-alpha.jar --scene C:\wrk\git\simdata\demo\src\main\resources\app-management-simple.json --simulation C:\wrk\git\simdata\demo\src\main\resources\app-management-simple.simulation

[Damien_Dallimor]

Post your config if you can.

1) jvm startup command with the agent
2) your properties file
3) any log messages