AppDynamics for JBoss AS 7.1.1

[CommunityUser]

Hi,

We are planning to configure AppDynamics for JBoss AS 7.1.1, but we are facing issues. After adding the startup arguments and providing controller info in controller-info.xml under conf directory of AppDynamics, the javaagent was able to come up, but the JBoss instance is failing to start.

If we just give the startup arguments and leave controller info, the agent was not coming up but the JBoss instance was able to start.

I am attaching the JBoss logs gathered while providing controller info and without controller info. Also attaching the agent logs when the agent was able to come up. Please advice.

Regards,

Vivek

[Arun_Dasetty]

Hi Vivek,

If we are concerned on why two agents are showing down please send the agent logs from affected nodes as from screenshot it is not clear what is the node and tier details in view, please provide screenshot with node name and tier name column in view for better clarity and to assist you further.

Regarding your other concern , we understood that you are seeing node names as  KANA_2--1 and KANA_2--2 format, and you cannot correlate with particular node represents which server prefixed by text string  KANA_2 if this is the case, we informed to do not provide hardcoded value in -javaagent entry -D jvm arg , we request you to pass the $serverVar for -Dappdynamics.agent.reuse.nodeName.prefix=  arg in addition prefix string such as below:

-Dappdynamics.agent.reuse.nodeName.prefix=KANA_2_$JVMInstanceName

and if $JVMInstnaceName environment Or variable property represents dynamic name of each instance that should make different here in node name convention , Please refer my earlier post we suggested to use dynamics variable name as variable followed by prefix if any expected for jvm arg.

Let us know if that information helps.

Regards,

Arun

[CommunityUser]

Arun,

When we first implemented the changes and restarted the servers we saw KANA-1, KANA-2,...., KANA-6. After we shutdown all the instances, all are gone except KANA-3 & KANA-4 even though they are completely shutdown. After restarting all the servers, we got KANA-7 and KANA-8 along with KANA-3 and KANA-4 in display. Is there any way we can get them removed. I tried to delete them, but permission denied for me.

Regarding -Dappdynamics.agent.reuse.nodeName.prefix=KANA_2_$JVMInstanceName, we will try to work on it and let you know.

Regards,

Vivek

[Arun_Dasetty]

Hi Vivek,

Can you see if user logged in have enough permissions under Setup -> Administration -> Users screen in edit view of user facing the issue and allocate required permissions if reuqired and save changes and if persists send the screenshot from UI and server.log for our reivew to assist you further?

Sure keep us posted. Regarding nodes in UI and re-apprearence it could be the case that you are viewing cached view the reuse property helps to re-use old node names instead creating new nodes when jvm reporting to old node say KANA-3 is down the new jvm will use same node name KANA-3 for which no agent is currently connect. Hope this is the case.

Regards,

Arun

[Arun_Dasetty]

Hi Vivek,

- from stdout.log_with_controller_info /stdout.log_without_controller_info we do no locate the jvm arg with com.singularity and other jvm args set in standalone.conf are not being picked up and hence we see the error listed below in logs:

JAVA_OPTS already set in environment; overriding default settings with values ..
10:47:16,487 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC00001: Failed to start service jboss.txn.ArjunaRecoveryManager: org.jboss.msc.service.StartException in service jboss.txn.ArjunaRecoveryManager: Failed to start service
..10:52:13,453 ERROR [org.apache.coyote.http11.Http11Processor] (http-kstllkand01-192.168.27.236-8280-2) Error processing request: java.lang.NoClassDefFoundError: com/singularity/ee/agent/appagent/entrypoint/bciengine/FastMethodInterceptorDelegator

- Can you take backup of standaline.sh and replace with standalone.sh in attachement and restart the jboss and see how it goes.

- Please be sure that we need to make sure the following args are shown in log stdout.log_with_controller_info on next jboss start up, Hope that information helps.

-Djboss.modules.system.pkgs=org.jboss.byteman,om.appdynamics,com.appdynamics.,com.singularity .. -Djava.util.logging.manager=org.jboss.logmanager.LogManager

Regards,

Arun

[CommunityUser]

Hi Arun,

Thanks for your help. I tried your config file, it worked. But the thing is as per the information on the website, we had those arguments in standalone.conf and only javaagent argument is standalone.sh. So, I guess it depends on the environment setup for using the required config file. Once again thank you so much.

Regards,

Vivek

[CommunityUser]

Hi Arun,

The app agent is working fine for single instance on a server in Development environment. In UAT & PROD, we have multiple standalone instances running on a single server which use the same standalone.sh config file from JBoss Home. How do we configure AppDynamics for all those instances on a single server. Is it something we need to configure in controller-info.xml?

I have attached our controller-info.xml for your reference. Please check it and let us know.

Note:

Below is the output for grepping one instance UAT_PK1. We have four more instances UAT_PK2, UAT_PK3, UAT_PK4 & UAT_PK5 on the same server, all of them using the same standalone.sh. Please let us know how do we setup the appdynamics for all the 5 instances.

chtrusr 38432 1 0 Nov12 ? 00:00:00 /bin/sh /charter/apps/kana/jboss/bin/standalone.sh --server-config=standalone-full-UAT.xml --properties=file:/charter/apps/kana/AgentDesktop/work/jboss/combined.1173544163.properties -Djboss.socket.binding.port-offset=100 -Djboss.server.base.dir=/charter/apps/kana/jboss/server/UAT_PK1 -Djboss.server.log.dir=/charter/apps/kana/share/logs/jboss/kstllkaau01/UAT_PK1 -Dcom.kana.properties.url=file:/charter/apps/kana/AgentDesktop/work/jboss/combined.1173544163.properties -Dcom.gtnet.systemProperties.override.url=file:/charter/apps/kana/AgentDesktop/config/UAT//override.properties -Dcore.home=/charter/apps/kana/AgentDesktop -Denvironment.dir=/charter/apps/kana/AgentDesktop/config/UAT -Denvironment.url=file:/charter/apps/kana/AgentDesktop/config/UAT/ -Drelease.id=1 -Dorg.apache.tomcat.util.http.Parameters.MAX_COUNT=1000 -Dorg.jboss.as.logging.per-deployment=false -b kstllkaau01

chtrusr 38476 38432 0 Nov12 ? 01:07:33 /charter/apps/kana/java/jdk1.7.0_51/bin/java -D[Standalone] -server -XX:+UseCompressedOops -Xms2048m -Xmx2048m -XX:MaxNewSize=256M -XX:NewSize=256M -XX:PermSize=256M -XX:MaxPermSize=512M -XX:AutoBoxCacheMax=2048 -XX:+DisableExplicitGC -Dsun.rmi.dgc.client.gcInterval=36000000 -Dsun.rmi.dgc.server.gcInterval=36000000 -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -Xloggc:/charter/apps/kana/share/logs/jboss/kstllkaau01/UAT_PK1/gc.20141112.1128.log -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.log.org.apache.commons.httpclient.wire=ERROR -Djavax.net.ssl.trustStore=/charter/apps/kana/AgentDesktop/config/UAT/UAT_truststore.jks -Djavax.net.ssl.trustStorePassword=Charter817 -Dorg.apache.coyote.http11.Http11Protocol.COMPRESSION=on -Dorg.jboss.boot.log.file=/charter/apps/kana/share/logs/jboss/kstllkaau01/UAT_PK1/boot.log -Dlogging.configuration=file:/charter/apps/kana/jboss-eap-6.0/server/UAT_PK1/configuration/logging.properties -jar /charter/apps/kana/jboss/jboss-modules.jar -mp /charter/apps/kana/jboss/modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=/charter/apps/kana/jboss -Djboss.server.base.dir=/charter/apps/kana/jboss-eap-6.0/server/UAT_PK1 --server-config=standalone-full-UAT.xml --properties=file:/charter/apps/kana/AgentDesktop/work/jboss/combined.1173544163.properties -Djboss.socket.binding.port-offset=100 -Djboss.server.base.dir=/charter/apps/kana/jboss/server/UAT_PK1 -Djboss.server.log.dir=/charter/apps/kana/share/logs/jboss/kstllkaau01/UAT_PK1 -Dcom.kana.properties.url=file:/charter/apps/kana/AgentDesktop/work/jboss/combined.1173544163.properties -Dcom.gtnet.systemProperties.override.url=file:/charter/apps/kana/AgentDesktop/config/UAT//override.properties -Dcore.home=/charter/apps/kana/AgentDesktop -Denvironment.dir=/charter/apps/kana/AgentDesktop/config/UAT -Denvironment.url=file:/charter/apps/kana/AgentDesktop/config/UAT/ -Drelease.id=1 -Dorg.apache.tomcat.util.http.Parameters.MAX_COUNT=1000 -Dorg.jboss.as.logging.per-deployment=false -b kstllkaau01

Regards,

Vivek

[Arun_Dasetty]

Hi Vivek,

Can you leave node-name element in controller-info.xml blank if you want to use same agent_dir but want to name each JVM instance using -javaagent arg a unique node dynamically and see how it goes after server restart with following -D jvm args in place:

-Dappdynamics.agent.reuse.nodeName=true -Dappdynamics.agent.auto.node.prefix=myPrefix

Let us know if that information helps

Regards,

Arun

[CommunityUser]

Hi Arun,

You just rock man. I can't thank you enough. We tried this change on one host out of 7. It picked all the 6 instances.

But another issue we are having is identifying the instances. In controller-info.xml we specified "<tier-name>KANA</tier-name>", and in the AppDynamics site we are seeing "KANA-1, KANA-2, KANA-3, KANA-4, KANA-5 & KANA6. We don't know which instance they are referring to since we start all the servers at a time. And also all the 6 instances are named similar across 7 hosts. So, if we want to see all the 42 instances under KANA in AppDynamics what needs to be done. I am attaching the standalone.conf, standalone.sh and controller-info.xml which we used for one host with 6 instances. 

Once again thanks for your help and support.

Regards,

Vivek

[Arun_Dasetty]

Hi Vivek,

Glad that our suggestions had helped. Regarding latest query, Please replace above jvm args with the following if you do not want to name node with tier name as prefix and want to name nodes with particular prefix:

 -Dappdynamics.agent.reuse.nodeName=true -Dappdynamics.agent.reuse.nodeName.prefix=testprefix

And if you want each node to identifier with some string that represents servername , you might need to identify the environment variable and provide that as value to jvm arg as below, check if that information helps.

Windows:
-Dappdynamics.agent.reuse.nodeName=true -Dappdynamics.agent.reuse.nodeName.prefix=%COMPUTERNAME%
Linux:
-Dappdynamics.agent.reuse.nodeName=true -Dappdynamics.agent.reuse.nodeName.prefix=$SERVERJVM_NAME

Please try whether env. variable picking up or not in test env. or test app and see how it goes.

Regards,

Arun