Splunk AppDynamics

JVM crash with AppD agent and G1GC

CommunityUser
Splunk Employee
Splunk Employee

We are facing JVM crash with AppD agent and G1GC, unable to find any possible root cause or solution to this. Anyone have the similar issue with a fix?

Labels (1)
0 Karma

Peter_Holditch
Builder

A JVM crash is always a result of a JVM bug (unless it is triggered by an issue in native code accessed through JNI.  The AppDynamics agent contains no native code) .  Of course, adding an agent to the JVM changes what is going on within the JVM (for instance, classes getting retransformed would not happen with no agent present) which can cause JVM bugs to be triggered which are not triggered in the absence of the agent.

Unfortunately, selecting g1gc seems to uncover more bugs of this nature than you might hope (often, coincident with class retransformations)

You need to see if the crash backtrace corresponds to a known JVM bug and if so, upgrade to a JVM where it is fixed.  Or select a different GC algorithm.  Or, you can disable dynamic class retransformations which does seem to avoid many of these issues (at the expense of not being able to change instrumentation on the fly)

Warm regards,

Peter

0 Karma

Arun_Prasad_Che
Explorer

Where and how do we "disable dynamic class retransformations"

Does this have any impact on what agent can monitor?

0 Karma

Vaibhav_Vir_Sin
Communicator
You may add below property to app-agent-config.txt file

-Dappdynamics.agent.disable.retransformation=true
0 Karma

CommunityUser
Splunk Employee
Splunk Employee

What does this options means?
Does it disable able agent on-the-fly configuration changes (ie, if we change BT definition, we will require application restart)?

If the JVM crash is caused by retransformation, which trigger the JVM bug.

Does AD agent reveal on log messages about the retransformation behavior?

0 Karma

CSI_AppD_Licens
Engager

What we are lossing adding flag? is this only for a particular agent version?

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Enhance Security Operations with Automated Threat Analysis in the Splunk EcosystemAre you leveraging ...

What Is Splunk? Here’s What You Can Do with Splunk

Hey Splunk Community, we know you know Splunk. You likely leverage its unparalleled ability to ingest, index, ...

Level Up Your .conf25: Splunk Arcade Comes to Boston

With .conf25 right around the corner in Boston, there’s a lot to look forward to — inspiring keynotes, ...