Splunk AppDynamics

Agent nodes replicating themselves

Hari_Shree_Josh
Explorer

We recently orchestrated an application on the dev environment and everything went well but now what is happening is that the nodes agent status keeps going to 0% and then they replicate themselves (within the tier). I have attached a SS that might give a clearer picture. 

These copies of java nodes get deleted from the controller after the application server is restarted but then the issue reoccurs.

My best guess is that there was something incorrectly done while deploying the agents and I was hoping if someone else would image.pngalso faced the same issue and might be able to tell me what could be wrong.

Thanks,

Hari Shree Joshi 

Labels (3)
1 Solution

Yusuke_Miyahara
Explorer

Hello Hari Shree,

Thank you for your request to our community. We are happy to assist you with this matter.


It seems you are running Java app in a dynamic environment. In dynamic environment, the vanilla agent in each container will register as a separate node, so such situation occurs. You need to add configuration in the dynamic environment as follows.

[JVM properties]
-Dappdynamics.agent.reuse.nodeName=true
-Dappdynamics.agent.reuse.nodeName.prefix=<YOUR PREFIX HERE>
-Dappdynamics.jvm.shutdown.mark.node.as.historical=true


You can also set the above parameter in your <agent-dir>/verx.x.xxx/conf/controller-info.xml or app-agent-config.xml.
Please refer the document below.
https://docs.appdynamics.com/display/PRO45/Java+Agent+Configuration+Properties#JavaAgentConfiguratio...
https://docs.appdynamics.com/display/PRO45/Instrument+JVMs+in+a+Dynamic+Environment#InstrumentJVMsin...


The last property is necessary for marking the node historical as soon as they are terminated.
Please refer the document below:
https://docs.appdynamics.com/display/PRO45/Instrument+JVMs+in+a+Dynamic+Environment#InstrumentJVMsin...


If you have already set the above properties and the problem still remains, please make sure whether your application environment abortively terminates JVMs rather than allowing them to shut down gracefully or not. Because in this case, the terminated node is not marked as historical. As a result, it sometimes happen that the node name is not reused.

Best regards,
Yusuke

View solution in original post

Yusuke_Miyahara
Explorer

Hello Hari Shree,

Thank you for your request to our community. We are happy to assist you with this matter.


It seems you are running Java app in a dynamic environment. In dynamic environment, the vanilla agent in each container will register as a separate node, so such situation occurs. You need to add configuration in the dynamic environment as follows.

[JVM properties]
-Dappdynamics.agent.reuse.nodeName=true
-Dappdynamics.agent.reuse.nodeName.prefix=<YOUR PREFIX HERE>
-Dappdynamics.jvm.shutdown.mark.node.as.historical=true


You can also set the above parameter in your <agent-dir>/verx.x.xxx/conf/controller-info.xml or app-agent-config.xml.
Please refer the document below.
https://docs.appdynamics.com/display/PRO45/Java+Agent+Configuration+Properties#JavaAgentConfiguratio...
https://docs.appdynamics.com/display/PRO45/Instrument+JVMs+in+a+Dynamic+Environment#InstrumentJVMsin...


The last property is necessary for marking the node historical as soon as they are terminated.
Please refer the document below:
https://docs.appdynamics.com/display/PRO45/Instrument+JVMs+in+a+Dynamic+Environment#InstrumentJVMsin...


If you have already set the above properties and the problem still remains, please make sure whether your application environment abortively terminates JVMs rather than allowing them to shut down gracefully or not. Because in this case, the terminated node is not marked as historical. As a result, it sometimes happen that the node name is not reused.

Best regards,
Yusuke

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  &#x1f680; Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...