Table of Contents Issue Enabling Flash Additional Recommendations Issue When clicking on certain screens in the Controller (e.g., Analytics → Configuration), you may encounter the message "Please install Flash to use this feature" in the top right corner. This is due to the fact that a few screens in the Controller UI leverage Flash and require Flash Player 10 or greater. AppDynamics recommends version 11.   This behavior may be noticed in all browsers, but the steps in this article are specific to Chrome.  We recommend that all customers using the latest version of Chrome enable Flash. Please note that it’s a top priority for AppDynamics to remove Flash from the product and work is currently in process. Watch for announcements in our monthly Product Updates. Enabling Flash In the latest versions of Chrome, your Flash settings will be overridden every time you restart your browser. Please follow the instructions below to enable Flash in your specific Chrome version: Chrome v70 Chrome v71 Chrome v70 If you are using Chrome v70, the steps below guide will you through how to enable Flash permanently so you don’t need to enable it each time you launch Chrome. 1. Open Chrome 2. Go to: chrome://flags/ 3. Search for: Enable Ephemeral Flash Permissions 4. Change its value to: Disabled 5. Restart Chrome 6. Go to: chrome://settings/content/flash 7. Click on Allow - Add button 8. Add site: https://[*.]saas.appdynamics.com 9. Restart Chrome Chrome v71 The Enable Ephemeral Flash Permissions feature mentioned above is not available in Chrome v71. If you quit Chrome, your Flash settings won’t be saved. You will need to re-enable Flash each time you load your browser. To enable Flash, follow the steps below: 1. Browse to your AppDynamics Controller. 2. Click the drop-down menu to the left of the URL bar (typically a lock icon). 3. Set Flash settings to Allow. (Note: If you don’t see the Flash setting, click Site Settings and you will see the option to enable Flash) 4. Reload the page. 5. If you close and restart the browser, you will need to complete these 4 steps above again. Additional Recommendations As a best practice, we also recommend disabling AdBlock when using the Controller UI to avoid issues loading particular screens. For instructions, see: How do I disable AdBlock in Chrome? ... View more

Question How do I instrument Apache Tomcat on CA Project Portfolio Management (Clarity PPM)? Solution Launch the NSA console  Navigate to CSA properties -> Application tab Under "Application Server" select Apache Tomcat as the vendor and edit the Java VM parameters as follows: Add AppDynamics Agent entry:   -javaagent:<agent_home>/javaagent.jar For more information: Load the Java Agent in a JVM   Last updated: 2/20/19 (updated product name) ... View more

After a v4.3 upgrade, some users may need to resolve an error AppDynamics v4.3 offers persistent view options for users using a POST request. Because some Java versions can't parse slashes ("/") in POST query parameters unless the "/" is defined in the config file, some users may see an error message. This article includes the steps to edit the config file with a defined "/". Table of Contents Problem Cause and Solution Additional Resoiurces Problem After upgrading or installing AppDynamics version 4.3.x, users may see the following error when accessing the "Servers" tab in the Controller UI. The requested URL /controller/restui/userPreferenceManager/setGeneralUserPreference/SIMMachinesListController-stateId--SIM_MODE-4.3-metricToggles/[{"selected":false,"label":"Availability (%)","type":0,"metricName":"Hardware Resources|Machine|Availability"},{"selected":false,"label":"Availability (%) Trend","type":1,"metricName":"Hardware Resources|Machine|Availability"},{"selected":true,"label":"CPU (%)","type":0,"metricName":"Hardware Resources|CPU|%Busy"},{"selected":true,"label":"CPU (%) Trend","type":1,"metricName":"Hardware Resources|CPU|%Busy"},{"selected":false,"label":"CPU Steal (%)","type":0,"metricName":"Hardware Resources|CPU|%Stolen"},{"selected":false,"label":"CPU Steal (%) Trend","type":1,"metricName":"Hardware Resources|CPU|%Stolen"},{"selected":true,"label":"Disk I/O (%)","type":0,"metricName":"Hardware Resources|Disks|Avg IO Utilization (%)"},{"selected":true,"label":"Disk I/O (%) Trend","type":1,"metricName":"Hardware Resources|Disks|Avg IO Utilization (%)"},{"selected":true,"label":"Disk Usage (%)","type":0,"metricName":"Hardware Resources|Volumes|Used (%)"},{"selected":true,"label":"Disk Usage (%) Trend","type":1,"metricName":"Hardware Resources|Volumes|Used (%)"},{"selected":false,"label":"Load - 1min","type":0,"metricName":"Hardware Resources|Load|Last 1 minute"},{"selected":false,"label":"Load - 1min Trend","type":1,"metricName":"Hardware Resources|Load|Last 1 minute"},{"selected":true,"label":"Memory (%)","type":0,"metricName":"Hardware Resources|Memory|Used %"},{"selected":true,"label":"Memory (%) Trend","type":1,"metricName":"Hardware Resources|Memory|Used %"},{"selected":false,"label":"Memory Swap Used (%)","type":0,"metricName":"Hardware Resources|Memory|Swap Used %"},{"selected":false,"label":"Memory Swap Used (%) Trend","type":1,"metricName":"Hardware Resources|Memory|Swap Used %"},{"selected":true,"label":"Network I/O (%)","type":0,"metricName":"Hardware Resources|Network|Avg Utilization (%)"},{"selected":true,"label":"Network I/O (%) Trend","type":1,"metricName":"Hardware Resources|Network|Avg Utilization (%)"}] was not found on this server. Following is a screenshot of the same error. Cause and Solution As of AppDynamics v4.3, view options are persistent for users. To do this, AppDynamics keeps track of the last tab a user selected on the time previous by using an API call. The cause of the error is that some versions of Apache are unable to parse slashes ("/") in POST query parameters unless the "/"  is defined as query parameter in the config file. Solution The solution is to turn on the AllowEncodedSlashes NoDecode  option in the Controller server's Apache config file ( httpd.conf ). Additional Resources For more information, see the Stack Overflow discussion Need to allow encoded slashes on Apache.   ... View more

When creating database custom query metrics, if the queries are complex, they may return a timeout error message in the Database Agent logs.  For example from DB2 database: Caused by: com.ibm.db2.jcc.am.SqlTimeoutException: DB2 SQL Error: SQLCODE=-952, SQLSTATE=57014, SQLERRMC=null, DRIVER=4.16.53 As a solution, a user can increase the timeout parameter by adding the following property in the DB Agent startup file, or as a system argument if DB Agent is started through a command prompt or terminal.  -Ddbagent.custommetric.query.timeout.seconds=<time_in_second>    Example: -Ddbagent.custommetric.query. timeout.seconds=120     ... View more

To set up a Java Agent to automatically give unique names to nodes for the application, add the following element to the /conf/controller-info.xml file. <auto-naming>true</auto-naming> Example using a 4.2.14.0 Agent: Add the following element to the /AppServerAgent-4.2.14.0/ver4.2.14.0/conf/controller-info.xml file: <auto-naming>true</auto-naming> Restart the JVM. Node names will be listed as null in the Controller UI until the server is started. Naming convention is  application_server@host . ... View more

How do I resolve a non-admin user permissions errors in ALD, OIT, or custom memory structures? Problem When a non-admin user runs automatic leak detections (ALD), object instance tracking(OIT), or Custom memory structures, they may see the following error: The current user 'ABC' does not have permission to do this. (action: CONFIG_AGENT_PROPERTIES entity: APPLICATION_COMPONENT_NODE 235) (agentManager.updateAgentConfiguration)   Solution To resolve a user's permission errors, an administrator must edit their agent permission and role properties. To determine the permission name, extrapolate it from the error message. For example, the error message shown above indicates a permission name of Configuration Agent Properties: Open the user's account permissions and edit the account to include that permission.  Administrator can add or remove user roles and permissions here     ... View more

When running WebSphere with Security Enabled, the user must provide permissions to the Java Agent, to ensure a seamless interaction between the two. Add the following snippet to the security policy file: grant codeBase "file:\* AGENT_DEPLOYMENT_DIRECTORY \*/-"   {      permission java.security.AllPermission; };   The security policy file is located in the AppDynamics Agent logs under JVM arguments:    -Djava.security.policy=...../security.policy Or Unix users can use the ps command to find the command line parameters for the JVM. If the JVM argument is not provided, the application may be using a default Java policy file ( java.policy ). Add the previously mentioned snippet to this file.    Examples where applying permissions is not sufficient: 1. Agent logs show: java.security.AccessControlException: access denied ("java.io.FilePermission" "/xx/xx/xx/y.xml" "read")   grant codeBase "file:\* AGENT_DEPLOYMENT_DIRECTORY \*/-"  {    permission java.security.AllPermission;    permission java.io.FilePermission "*","read,write,execute";  };   2. Agent logs show:  java.security.AccessControlException: access denied (java.lang.RuntimePermission getClassLoader)   grant codeBase "file:\* AGENT_DEPLOYMENT_DIRECTORY \*/-"  {  permission java.security.AllPermission;  permission java.lang.RuntimePermission "getClassLoader"; };     3. Agent logs show: java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "findMBeanServer")   grant codeBase "file:\* AGENT_DEPLOYMENT_DIRECTORY \*/-"  {  permission java.security.AllPermission;  permission javax.management.MBeanServerPermission "findMBeanServer"; };  Note: Replace  \*AGENT_DEPLOYMENT_DIRECTORY\* with the Agent home directory path. Do not miss the forward slash (/-) at the end.    Linux example:  grant codeBase "file:/opt/appdynamics/appserveragent/-" Windows example:  grant codeBase "file:c:/appdynamics/appserveragent/-"                    Additional solutions:    Add the following property as a JVM argument only on development instances. Remove this property as soon as security debugging is completed.  -Djava.security.debug="access,failure"   Additionally, create a node level property, which can coincide with the above security policy file changes.   Node property example: property name: enable-interceptors-for-security type: boolean value: true   Note: Changes in the security policy file require a JVM restart.    ... View more

To configure Apache Solr version 6.5.x with AppDynamics Agent (preferably version 4.2.x or higher), use the following steps: Navigate to /<solr-home>/bin/solr.in.sh on the Controller machine. Add the following snippet to the solr.in.sh file in order to add the Java Agent. SOLR_OPTS="$SOLR_OPTS -javaagent:<app_agent_install>/ver4.2.10.0/javaagent.jar" Example: SOLR_OPTS="$SOLR_OPTS -Dappdynamics.agent.applicationName=Solr -Dappdynamics.agent.tierName=SolrTier -Dappdynamics.agent.nodeName -javaagent:/Users/XXX/Documents/Agent/AppServerAgent-4.2.10.0/ver4.2.10.0/javaagent.jar"   Restart Solr. ... View more

Scenario: A legacy Machine Agent and Java Application Agent are installed on a machine. Despite correct installation and configuration, one or both of the Agents do not report. The first step in troubleshooting this issue is to validate the Unique Host ID, which is a problem more common with Machine Agents. Machine Agents use the Java API to get the Unique Host ID. Sometimes the same JVM can return different values for the same machine each time the Machine Agent is restarted. One possible cause is the Unique Host ID of the App Agent and the Machine Agent are not in sync. In this case, the Machine Agent does not know which application to register. Validate: Search for the application or the Unique Host ID on both the Application and Machine Agents, and make sure that the values are the same. Examples:  To confirm the issue, go to the Agent settings page located at either of the following URLs:   http://<host:Port>/controller/#/location=SETTINGS_AGENTS http://<host:Port>/#/location=SETTINGS_MACHINE_AGENTS  Also, verify that the Machine Agent is associated with the application.      Solution: If there is any discrepancy between the two Agents, resolve the issue by providing the same Unique Host ID value for the App Agent and the Machine Agent using the following steps. Note: If it is not feasible to stop the App agent, it is possible to sync the Unique Host ID value of the Machine Agent with the App Agent, and then apply the property to the App Agent on the next restart. Stop both Agents. Start the App Agent with the following property: -Dappdynamics.agent.uniqueHostId=<your-host-id> system property Confirm that there are no new errors or issues with the App Agent. Start the Machine Agent with the same property: -Dappdynamics.agent.uniqueHostId=<your-host-id> system property Another option is to add a <unique-host-id>your-host-id</unique-host-id> element within the <machine_agent_install>/conf/controller-info.xml file. ... View more

Custom metrics allows the user to specify a SQL query to run on a monitored DB every minute, and collect the results of the query. This document specifies restrictions and limitations, as well as some examples and debugging techniques.  Restrictions and Limitations: The query must always return a positive integer number. The query should not return more than one column. The user can define up to 20 custom metrics per monitored DB. This limit is configurable through admin.jsp with the following property; however, it should be increased graciously. For Security reasons, we only allow SELECT queries. How to debug?   If the Agent encounters an error while running the query, the Agent will send an error event to the Controller. View these error events by clicking on Database -> Events in the Controller UI. For more details on an error, view the database Agent logs located at <db_install_dir>/logs . The errors are self-descriptive and can be searched using the keywords "CustomMetricsCollector," "ERROR," or "WARN." Examples: [<MyTestMysql>-Scheduler-3] 29 May 2017 11:04:59,702 ERROR CustomMetricsCollector - Error collecting data for database 'MyTestMysql' java.sql.SQLException: Invalid custom metric: test4 , actual data type is non-numeric and expected data type is numeric (Custom Metric : test4) 09 Jun 2017 13:40:08,889 ERROR [<Database>-Scheduler-1] CustomMetricsCollector:130 - Error collecting data for database 'Database' java.sql.SQLException: DB2 SQL Error: SQLCODE=-952, SQLSTATE=57014, SQLERRMC=null, DRIVER=4.16.53 (Custom Metric : TestMetric) Query Examples with MySQL: 1.select COUNT(1) "total" from dual; - Worked Output: 1 2. select 9+ ' ' +':00' num from dual; - Worked. Output: 9  3. select name from user where id=3 - Not working as expected Output: root 4. select id as node_id from application_component_node where id NOT in (select distinct(node_id) from metricdata_hour where ts_min > ((select max(ts_min) from metricdata_hour)) - 60*2) - Worked/Not Worked based on output Output: Fails if returns multiple records   Related Links: Database monitoring metrics Configuring custom metrics ... View more

Linux users can use the following steps to determine which of their Java threads is consuming the most CPU. From your computer terminal: 1. First, find the Java process ID. ps -ef |grep java 2. Use the Java process ID (PID) to pull the lightweight processes into a file named lwp.txt . ps -eLo pid,lwp,nlwp,pcpu,etime,args|grep <pid> > lwp.txt 3. Use the PID to generate a Java thread dump. kill -3 <pid> 4. Choose the lightweight process (LWP) that is consuming the most CPU from the lwp.txt file created in step 2, and convert the LWP ID from decimal to hexadecimal using the tool of your choice (for example, binaryhexconverter.com). Example: Convert LWP DEC 4235 to HEX 108B. 5. Search the thread dump created in step 3 for the hexadecimal LWP ID to find the Java thread that is consuming the most CPU. 6. If the problematic Java thread is part of AppDynamics, contact support for further assistance. If it is an application thread, contact your development/application team to debug the issue further. ... View more

Use the following recommendations if high CPU usage occurs after attaching the AppDynamics agent: Disable aggressive slow snapshot collection Exclude specific hotspot interceptors Async Instrumentation Disable unwanted instrumentation from transaction detection Turn off Turbo custom exit point interceptor 1. Disable aggressive slow snapshot collection If aggressive slow snapshot collection is enabled, the agent will retain selected call graph segments that precede the detection of a slow, stall or error transaction condition. Disabling the aggressive slow snapshot collection will significantly reduce overhead. In the Controller UI, click on Configuration in the left navigation bar. Click the double arrows in the top right to expand the menu. Click on Call Graph Settings. Uncheck the checkbox next to "Enable Aggressive Slow Snapshot Collection." Click the save button.    2. Exclude specific hotspot interceptors  Editing the node level property "exclude-interceptors" to exclude hotspots will reduce CPU usage. To determine if your agent has enabled hotspots, search the agent.* log files for "Enabled hotspots" and search the ByteCodeTransformer*.log for "Applying method interceptor diag.snapshot.BoundHotspot." Edit a node property: In the controller UI, click on Tiers and Nodes in the left navigation bar. Double click on the node you'd like to configure, and click Agents > App Server Agent > Configure. This will open the App Server Configuration window.  At the node level, click on the "Use Custom Configuration" button. Either search for the exclude-interceptors property and double click on it, or click on the gray plus sign to create a new agent property if the exclude-interceptors property does not already exist. Click the save button after adding the new configuration. A "saved successfully" message will appear in the top right corner of the window.  Name - exclude-interceptors Description - exclude-interceptors Type - String Value - com.singularity.BoundHotspotInterceptor Note: If there are already some class names added to this property, use a comma between them. Example: 3. Async Instrumentation Too many async interceptors can cause high CPU utilization. You may have unnecessary application framework classes instrumented that are not needed for visibility or monitoring. Get a list of classes/methods which are applying async interceptor by looking at the  ByteCodeTransformer*.log or search "async.handoffAsyncHandOffExecutionTracker." Exclude the packages 1-1 to improve CPU utilization. The package/class can be excluded either from fork-config OR using node property. Fork-config changes (recommended) - Requires JVM restart. Navigate to <agent_install_dir>/<ver4.X.x>/conf/app-agent-config.xml . Search for <fork-config> keyword. Add the desired packages to be excluded.  Example: <excludes filter-type="STARTSWITH" filter-value="<package/>"/> or fully qualified class name Few Examples: <excludes filter-type="STARTSWITH" filter-value="com.arjuna/"/> <excludes filter-type="STARTSWITH" filter-value="com.netflix/"/> <excludes filter-type="STARTSWITH" filter-value="com.bea/,com.weblogic/,weblogic/,com.ibm/,net/sf/,com/mchange"/> Node Property - See step 2 to create or edit a node property.   Name: thread-correlation-classes-exclude Description: thread-correlation-classes-exclude Value:Type: <fully.qualified.package.ClassName>,<package.name>.* Note: Excluding packages and classes could result in loss of visibility.    4. Disable unwanted instrumentation from transaction detection Spring Bean and EJB interceptors are CPU intensive. If you are not looking for any spring or EJB entry points, disable Transaction Detection.  From the controller UI, click on Applications. Click Configuration in the left navigation. Under the Transaction Detection tab, uncheck both of the checkboxes next to Spring Bean and EJB to disable them. 5. Turn off Turbo custom exit point interceptor  Turbo exit points are special interceptors that handle a high volume of calls. To determine if these interceptors are causing high CPU usage, search the  ByteCodeTransformer*.log for "exit.TurboCustom." If the classes are not adding visibility, exclude them and disable the interceptors by creating node-level properties (see "Edit a node property" in step 2).   Name - exclude-interceptors Description - exclude-interceptors Type - String Value - com.singularity.TurboCustomExitPointInterceptor Name - disable-ootb-turbo-interceptors Description - disable-ootb-turbo-interceptors Type - boolean Value - true If these troubleshooting recommendations do not improve your high CPU usage, contact our support team for further assistance. ... View more