AppServerAgentLite 3.3.4.0 with Tomcat 7

[CommunityUser]

I've installed the Lite version and the AppDynamics files all appear to startup fine. I'm able to login to the viewer and it sees the connection to the agent, but since adding the javaagent.jar file to my startup file (catalina.sh), my code no longer is processing. We have an external webpage for testing whether our code is processing and it fails immediately.

We are using Tomcat 7.0.26 on Linux OS (CENTOS 2.6.32-220.17.1.el6.x86_64) and jdk1.7.0_07

(catalina.sh)

if [ -z "$LOGGING_MANAGER" ]; then
JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
else
JAVA_OPTS="$JAVA_OPTS $LOGGING_MANAGER"
fi

# Load AppDynamics application performance monitoring
if [ "$1" = "start" -o "$1" = "run" ]; then
export JAVA_OPTS="$JAVA_OPTS -javaagent:/usr/local/AppServerAgentLite-3.3.4.0/javaagent.jar"
fi

(/var/log/tomcat/catalina.out)

Install Directory resolved to[/usr/local/AppServerAgentLite-3.3.4.0]
[INFO]: JavAgent - Using Agent Version [Server Agent Lite v3.3.4.0 RC Build Date 2013-03-20 20:23]
[INFO]: JavAgent - Running IBM Agent [No]
[INFO]: AgentInstallManager - AppDynamics Agent edition [lite]
[INFO]: AgentInstallManager - Appdynamics lite agent is setup with viewer at host [127.0.0.1], port [8990]. You may specify the viewer host and port by setting the system property 'appdynamics.viewer.host', 'appdynamics.viewer.port'
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver is running
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using system property [appdynamics.agent.applicationName] for application name [Server App]
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using system property [appdynamics.agent.tierName] for tier name [Server Tier]
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using system property [appdynamics.agent.nodeName] for node name [Server Node]
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using application name [Server App]
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using tier name [Server Tier]
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using node name [Server Node]
[INFO]: AgentInstallManager - Full Agent Registration Info Resolver finished running
[INFO]: AgentInstallManager - Agent runtime directory set to [/usr/local/AppServerAgentLite-3.3.4.0]
[INFO]: AgentInstallManager - Agent node directory set to [Server Node]
[INFO]: JavAgent - Agent Directory [/usr/local/AppServerAgentLite-3.3.4.0]
Agent Logging Directory [/srv/web/logs/AppServerAgentLite/Server Node]
Running obfuscated agent
Started AppDynamics Java Agent Successfully.

[Arun_Dasetty]

Hi Bill,

Can you please provide the following to debug further:
- zipped version of lite agent logs under /usr/local/AppServerAgentLite-3.3.4.0/logs directory

- complete stack trace for the scenaro in reference to your query "my code no longer is processing. We have an external webpage for testing whether our code is processing and it fails immediately." or error screenshot

- latest catalina logs under <tomcat>/logs directoy if any exceptions related to webapp access failed scenario

- tomcat console error stack trace depicting the webapp access failed scenario
 

Thanks,
Arun

[CommunityUser]

Attached Zipfile contains logs from /usr/local/AppServerAgentLite-3.3.4.0/logs directory, the catalina.out logfile and example pdf files of a good test (Pinger_good.pdf) and bad test (Pinger_bad.pdf).
I also included a "ps -ef | grep javaagent" output (ps_javaagent.txt) to show how it appears on the system.
Sorry, I don't have a stack trace of a failure. Everything appears to start fine but when we perform our self-test (the Pinger outputs) we get an immediate failure on the Pinger page.

[Arun_Dasetty]

Hi Bill,

Somehow i am not able to see logs attached, Can you please send the logs in your post to akumar@appdynamics.com to debug further

Thanks,

Arun

[Arun_Dasetty]

Hi Bill,

We Apologize for the delay in response. Thanks for the logs, please find the analysis of the logs:
1) From the catalina.out, we see that tomcat started fine successfully with agent:
----------------------------------------------------------

Registered app server agent with Node ID[1] Component ID[1] Application ID [1]
Started AppDynamics Java Agent Successfully.
May 24, 2013 11:10:54 AM org.apache.catalina.core.AprLifecycleListener init
..

May 24, 2013 11:11:57 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 63309 ms
[Full GC 347393K->210823K(3106944K), 1.5360590 secs]
------------------------------------------------------

2) From the BT.log we see that only the following BT got registered or invokded:

GCS-http-2] 24 May 2013 11:12:05,460  INFO - BT [1] Name [/OBN/Pinger.jsp] Type [SERVLET] Custom [false]

3) In reference to files Pinger_xx.pdf for OBN Server URL : http://localhost:80/OBN/OBNServlet , from BCI.log we do see that the OBNServlet servlet getting instrumentated successfully:

Applying method interceptor entry.Servlet (through entry.FastProxy) at com/garmin/obn/server/OBNServlet.doGet ((Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V) id:128
Applying method interceptor entry.webservice.cxf.Service (through entry.FastProxy) at org/apache/cxf/interceptor/ServiceInvokerInterceptor.handleMessage ((Lorg/apache/cxf/message/Message;)V) id:177

4) We see the following error in agent.log, we suspect you do see the verifyerrror in tomcat console error.log or tomcat console log, please let us know if any disconnect:
------------------------------------------
[AD Thread Pool-Global1] 24 May 2013 11:12:02,303  INFO ErrorProcessor - Error Objects registered with controller :{java.lang.VerifyError:=1}

-------------------------------------------

If the above said is right, With Java 7 you will need to prepend the following option before any of the JVM properties you are using and this issue is already addressed in our 3.6.6 pro version:

-XX:-UseSplitVerifier

This will disable the new type checker with StackMapTable attributes. Please ref. this link for more details about JVM option http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
 
 Ex:
 Before update:
 /opt/java/bin/java -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties ... -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -javaagent:/usr/local/AppServerAgentLite-3.3.4.0/javaagent.jar  -Dcatalina.base=/opt/tomcat -Dcatalina.home=/opt/tomcat -Djava.io.tmpdir=/opt/tomcat/temp org.apache.catalina.startup.Bootstrap start
 
 After update:
 /opt/java/bin/java -XX:-UseSplitVerifier -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties ... -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -javaagent:/usr/local/AppServerAgentLite-3.3.4.0/javaagent.jar  -Dcatalina.base=/opt/tomcat -Dcatalina.home=/opt/tomcat -Djava.io.tmpdir=/opt/tomcat/temp org.apache.catalina.startup.Bootstrap start

If you still see the issue, please send the error stack trace that you got in tomcat console

Thanks,

Arun