AppD Archive

After installing Java agent and Machine agent on tomcat node, not seeing activity in Controller

CommunityUser
Splunk Employee
Splunk Employee

Hi,

I'm using a AppDynamics Pro trial. I'm using an on-prem controller. I set up a Hello World JSP. On they tomcat node I installed the Java agent and the machine agent. I'm, not seeing any activity in the Controller.

I defined my own application name and tier. For the Java agent, I've added these to controller-info.xml:

    <controller-host>appd</controller-host>

    <controller-port>8090</controller-port>

    <controller-ssl-enabled>false</controller-ssl-enabled>

    <application-name>MyHelloApp</application-name>

    <tier-name>AppServerTier</tier-name>

    <node-name>tomcat-node</node-name>

 

For the machine agent, I've added this:

    <controller-host>appd</controller-host>

    <controller-port>8090</controller-port>

    <controller-ssl-enabled>false</controller-ssl-enabled>

 

I see that tomcat and the machine agent JVM both have established connections to my controller on port 8090.

 

On the controller, I see MyHelloApp. It has no business transactions defined. Under AppServer, I see my AppServerTier and my tomcat-node.

 

Under hardware, I get "No Data Available".

 

What am I doing wrong?

 

BTW I see there are 2 controller-info.xml files in the Java agent folder:

    conf/controller-info.xml

    ver4.0.6.0/conf

 

Which one should I be using?

 

Jim

 

0 Karma
1 Solution

Arun_Dasetty
Super Champion

Hi Jim,

Good to hear some load in UI now, Looks attachments are missing, let us know if you need further assistance. we see either deploying wars and generating load on missing wars could be reason which resolved post deploying apps and new load made

View solution in original post

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Jim,

For the server agent conig file, you want the one in the:

 ver4.0.6.0/conf

directory. Your settings look reasonable.

As for the machine agent, it could be a couple things, but can you let us know what OS you're using?

Thanks,

-Brad

0 Karma

CommunityUser
Splunk Employee
Splunk Employee
It's Centos 6.5

# uname -a

Linux gp1-int 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013
x86_64 x86_64 x86_64 GNU/Linux

0 Karma

Arun_Dasetty
Super Champion

Hi Jim,

 

Can you confirm and provide details requested below:

 

- As long as you are seeing node and appserver agent node jvm/heap data and machine agent reporting status under Node dashboard -> Agents screen up and fine we could confirm connectivity and registration of nodes are fine, let us know if appserver agent node under tomcat node dashboard -> Agents screen is not up and running? 

 

- if yes please send the screenshots from Controller UI in addition to archive version of logs folder under <Agent_dir>/<ver4.0.6.0>/logs and <machineagent_dir>/logs 

 

- We would like to check whether any ootb interceptors like servlet, struts etc as such are intercepted if agent connectivity is fine, if not the agent logs and should show some error in reference to connecitivity error, keep us posted the logs requested to assist you further.

 

If connectivity is fine, and no BTs registered , we might need to define custom pojo rule under Configure -> Instrumentation -> Transaction detection.

 

- For machine agent if default java monitor hava issue in getting h/w stats , we might need to switch to hardwaremonitor under <machienagent_dir>/monitors/ folder 

 

Regards,

Arun

0 Karma

CommunityUser
Splunk Employee
Splunk Employee
Attached are screenshots, and logs.

Thanks!

0 Karma

Arun_Dasetty
Super Champion

Hi Jim,

 

We do not locate any attachments, please provide requested logs to assist you further on this.

 

Regards,

Arun

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

ANd some screenshots...

0 Karma

Arun_Dasetty
Super Champion

Hi Jim,


- I confirm we see BT registered find at controller UI for app "MyHelloApp", Can you select last 3 days in ""Business transactions" screen , make sure no filters
in BT grid is selected and see whether you see some load in UI?

[http-8080-1] 02 Jun 2015 18:15:24,544 INFO BusinessTransactionRegistry - Found pre-registered business transaction: 172 for /HelloWorldJSP/helloWorld.jsp SERVLET

- Regarding the missing stats for machine agent, we do not see any suspicious information in logstomcat-node
Can you confirm whether you do not see laod under "tomcat-node" node dashboard -> Hardware section for last 3 days data at contorller UI

Regards,

Arun

0 Karma

CommunityUser
Splunk Employee
Splunk Employee
Hi Arun,

Under Business Transactions I see no entries at all, whether I look at last
15 minutes or 3 days or 1 year.

As for tomcat-node -> Hardware, the CPU utilization shows 0% even though
I'm running a busy script that is using 100% of the CPU.

I am attaching more screen shots. I do see that the agents are registered.

Jim

0 Karma

Arun_Dasetty
Super Champion

Hi Jim,

 

It looks logs are missing, Can you provide view of UI from similar section as referred below from you app?

image.png

 

We understood that CPU section shows zero % , Can you confirm whether memory and disk i/o are also zero ? 

If you do not see all hardware metrics as zero though under node dashboard -> agents section machine agent is up and reporting fine, please restart as below:

 

- Edit /opt/AppDynamics/machine-agent/monitors/JavaHardwareMonitor/monitor.xml and set <enabled> xml element value from true to false and save the changes to xml file

- Edit /opt/AppDynamics/machine-agent/monitors/HardwareMonitor/monitor.xml and set <enabled> xml element value from false to true and save the changes to xml file

 

Now restart machine agent java process and see how it goes.

 

- Also if no BTs shows up in UI , generate load on URI on tomcat server /HelloWorldJSP/helloWorld.jsp multiple times and see whether you see load in UI?

 

- If issue still persists send the server.log under <Controller_install_dir>/logs path and latest agent logs and machine agent logs and screenshots from controller UI for review

 

Regards,

Arun

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

 Hi,

It appears now I'm seeing CPU utilization at 100%. For some reason the
machine agent seems to be working now. I didn't change anything.

As for BT's, I now have a while loop running from my laptop:

curl -X GET http://gp1-ext:8080/HelloWorldJSP/helloWorld.jsp

I still see no BT's at all.

Here is a BT log from several days ago - is this normal?

# cd /opt/AppDynamics/agent/ver4.0.6.0/logs/tomcat-node

# cat BusinessTransactions.2015_06_02__18_15_09.0.log

[Thread-0] 02 Jun 2015 18:15:09,644 INFO - Starting BT Logs at Tue Jun 02
18:15:09 EDT 2015

[Thread-0] 02 Jun 2015 18:15:09,644 INFO -
###########################################################

[Thread-0] 02 Jun 2015 18:15:09,644 INFO - Using Java Agent Version
[Server Agent v4.0.6.0 GA #2015-05-11_20-44-33
r7cb8945756a0779766bf1b4c32e49a96da7b8cfe 10-4.0.6.next]

[Thread-0] 02 Jun 2015 18:15:09,644 INFO - Running IBM Java Agent [No]

[root@gp1-int tomcat-node]# cat
BusinessTransactions.2015_06_02__18_15_12.0.log

[Thread-4] 02 Jun 2015 18:15:12,416 INFO - Starting BT Logs at Tue Jun 02
18:15:12 EDT 2015

[Thread-4] 02 Jun 2015 18:15:12,416 INFO -
###########################################################

[Thread-4] 02 Jun 2015 18:15:12,416 INFO - Using Java Agent Version
[Server Agent v4.0.6.0 GA #2015-05-11_20-44-33
r7cb8945756a0779766bf1b4c32e49a96da7b8cfe 10-4.0.6.next]

[Thread-4] 02 Jun 2015 18:15:12,416 INFO - Running IBM Java Agent [No]

[http-8080-1] 02 Jun 2015 18:15:24,545 INFO - BT [1] Name
[/HelloWorldJSP/helloWorld.jsp] Type [SERVLET] Custom [false]

[http-8080-1] 02 Jun 2015 18:15:24,549 INFO - Discovery Stack

Thread Name:http-8080-1

ID:61

Time:Tue Jun 02 18:15:24 EDT 2015

State:RUNNABLE

Priority:5


org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java)

javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377)

org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)

org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)

javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)

org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)

org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)

org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)

org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)

org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)

org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)

org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)

java.lang.Thread.run(Thread.java:745)

I am attaching a file with other logs from the controller.

I also attached a screenshot of BTs. How do I get the 

 

0 Karma

Arun_Dasetty
Super Champion

Hi Jim,

 

We see no issues in the screenshot, and we request you to refer the same screen for which you have provided screenshot , where we can see "showing 0 of 1 transactions" , request you to uncheck filters and select last 3 week time and you should see BT

 

You have referred in logs.txt from controller internal application which is not required for this, Let us know if that clarifies.

0 Karma

CommunityUser
Splunk Employee
Splunk Employee
While trying to get EUE working and bringing in a new war file, Business
Transactions started working. Weird. See attached.


0 Karma

Arun_Dasetty
Super Champion

Hi Jim,

Good to hear some load in UI now, Looks attachments are missing, let us know if you need further assistance. we see either deploying wars and generating load on missing wars could be reason which resolved post deploying apps and new load made

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Let me try again. Here are the logs

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...