Why do I need to collect the debug-level log file? The Cluster Agent by default logs the entries at the info level. Sometimes, the debug-level log files are necessary to investigate an experienced issue. Debug-level logging logs are more insightful entries that can be later used to identify the root cause of the experienced issue. There are two ways you can collect the agent log files at the desired logging level. By editing the currently running Cluster Agent By updating the Cluster Agent with a new configuration. 1. Edit the currently running Cluster Agent Determine the clusteragent object name: kubectl -n appdynamics edit clusteragent -n appdynamics Edit the YAML file as shown below to set the logLevel to DEBUG, then save it (similar to using the vi editor): kubectl edit clusteragent <cluster_agent_name> -n appdynamics Edit the file to include: Save the configuration: `:wq` Delete Cluster Agent pod with old configuration: kubectl delete pod <cluster-agent-pod> -n appdynamics Now cluster should spin up a new cluster agent pod. Keep the cluster-agent running for 15 minutes in debug logging mode. Collect logs by executing: kubectl logs <cluster-agent-pod> -n appdynamics > cluster-agent.log Revert the changes 2. Update the Cluster Agent with a new configuration. Edit the configuration: a) Kubernetes CLI installation -> "cluster-agent.yaml" file b) Helm Chart installation -> "values.yaml" file Apply the changes a) Kubernetes CLI installation -> kubectl apply -f cluster-agent.yaml b) Helm Chart installation -> helm upgrade <release_name> -f values.yaml appdynamics-cloud-helmcharts/cluster-agent -n appdynamics Now cluster should spin up a new cluster agent pod. Keep the cluster-agent running for 15 minutes in debug logging mode. Collect logs by executing: kubectl logs <cluster-agent-pod> -n appdynamics > cluster-agent.log Revert the changes   I hope this article was helpful. Feel free to ask in case of any questions. ... View more

Step-by-Step Guide to Verifying SSL Certificates Using AppDynamics Cluster Agent with Target Allocator In this article, you will learn how to check if there are issues with the SSL Certificate for the On-Premises Controller using the Cluster Agent with Target Allocator. Below are the steps you can independently verify in the cluster. Step-by-Step Guide:   Secret Check if a Secret containing the certificate named "custom-ssl.pem" is created. Cluster Agent Check if the Cluster Agent configuration file contains a parameter with the name of the Secret containing the certificate: "cluster-agent.yaml" -> Kubernetes CLI "values.yaml" -> Helm Chart Check if the Cluster Agent pod description contains: kubectl describe pod <cluster_agent_pod_name> -n appdynamics Environment variable "APPDYNAMICS_CUSTOM_SSL_SECRET: ssl-cert" Volume mount Volume "agent-ssl-cert" Check the presence of "/opt/appdynamics/ssl/custom-ssl.pem" in the Cluster Agent container. kubectl exec -it <cluster_agent_pod_name> -n appdynamics -- /bin/sh   Target Allocator Check if the Target Allocator pod description contains: kubectl describe pod <target_allocator_pod_name> -n appdynamics Environment variable "APPDYNAMICS_CUSTOM_SSL_SECRET: ssl-cert" Volume mount Volume "agent-ssl-cert" Check the presence of "/opt/appdynamics/ssl/custom-ssl.pem" in the Target Allocator container. kubectl exec -it <target_allocator_pod_name> -n appdynamics -- /bin/sh   I hope this article was helpful. Let me know in the comments if you have any questions or if something isn't working.  ... View more

Resolving AppDynamics Auto-Instrumentation Challenges in Capsule Tenant This article provides a step-by-step guide to replicating a local environment using Capsule, focusing on resolving common issues with AppDynamics auto-instrumentation within Kubernetes namespaces. It covers the installation of Capsule, tenant creation, and namespace setup, followed by troubleshooting symptoms, diagnosis, and solutions. The guide is designed for technical community members seeking to optimize their Capsule and AppDynamics integration. How to replicate locally Install Capsule https://github.com/projectcapsule/capsule/blob/main/charts/capsule/README.md Follow the instructions to create your first tenant https://capsule.clastix.io/docs/general/getting-started#login-as-tenant-owner Create the "appdynamics" and "oil-production" namespaces; they will be attached to the same Capsule Tenant. Symptoms The Cluster Agent cannot auto-instrument any application. Diagnosis There are no AppDynamics annotations in the Deployment/StatefulSet description.   There is no AppDynamics annotation in the namespace description.   In the Cluster Agent debug logs, many errors are related to the Cluster and Capsule tenant configuration. A single ERROR and 'Adding instrumentation task' are repeated every 5 minutes. The same situation occurs in the customer environment. [ERROR]: 2025-04-03 08:56:14 - utils.go:1086 - Unable to update namespace oil-production after 4 retries [DEBUG]: 2025-04-03 08:56:14 - deploymenthandler.go:623 - Adding instrumentation task for deployment oil-production/java-app-to-instrument [DEBUG]: 2025-04-03 08:56:14 - instrumentationconfig.go:794 - rule default|java|oil-production matches Deployment java-app-to-instrument in namespace oil-production with labels map[appd_agent:enable] [DEBUG]: 2025-04-03 08:56:14 - instrumentationconfig.go:805 - Found a matching rule {default|java|oil-production .* map[appd_agent:[enable]] java first JAVA_TOOL_OPTIONS map[agent-mount-path:/opt/appdynamics alpine-init-container-image: image:docker.io/appdynamics/java-agent:25.2.0 image-pull-policy:Always] map[bci-enabled:true port:3892] 0 0 false false false false map[] map[] map[] map[] 0 false [] true} for Deployment java-app-to-instrument in namespace oil-production with labels map[appd_agent:enable] [DEBUG]: 2025-04-03 08:56:14 - instrumentationconfig.go:338 - Instrumentation state for Deployment java-app-to-instrument in namespace oil-production with labels map[appd_agent:enable] is true​   Solution Cluster Agent service accounts require additional privileges in the Capsule Tenant: Original Tenant Tenant with additional privileges for Cluster Agent service accounts. All roles available here:  https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles I hope this article was helpful ... View more

Step-by-Step Guide to Verifying SSL Certificates Using AppDynamics Cluster Agent In this article, you will learn how to check if there are issues with the SSL Certificate for the On-Premises Controller using the Cluster Agent. Below are the steps you can independently verify in the cluster.   Next steps: Check if a Secret containing the certificate named "custom-ssl.pem" is created. Check if the Cluster Agent configuration file contains a parameter with the name of the Secret containing the certificate: "cluster-agent.yaml" -> Kubernetes CLI "values.yaml" -> Helm Chart Check if the Cluster Agent pod description contains: kubectl describe pod <cluster_agent_pod_name> -n appdynamics Environment variable "APPDYNAMICS_CUSTOM_SSL_SECRET: ssl-cert" Volume mount Volume "agent-ssl-cert" Check the presence of "/opt/appdynamics/ssl/custom-ssl.pem" in the Cluster Agent container. kubectl exec -it <cluster_agent_pod_name> -n appdynamics -- /bin/sh I hope this article was helpful  ... View more

What does this error mean? We usually observe the log message below in the Application startup logs when the agent is unable to connect with the controller to retrieve the nodeName (in the case of using reuse.nodeName).  Started AppDynamics Java Agent Successfully. [Thread-0] Tue Apr 02 09:46:04 UTC 2019[INFO]: JavaAgent - Started AppDynamics Java Agent Successfully. 2019-04-02 09:46:09,545 ERROR Recursive call to appender Buffer 2019-04-02 09:46:09,547 ERROR Recursive call to appender Buffer Next steps: Could you please check if any logs are generated under the /opt/appdynamics-java/ver.xxx.xx/logs/  directory and share them if available? If there are no logs, please add the configuration line below under the instrumentatioRules applied to the problematic pod:" customAgentConfig: -Dappdynamics.agent.reuse.nodeName=false -Dappdynamics.agent.nodeName=test   If you are using Cluster Agent version >=23.11.0, to force re-instrumentation, you need to use the additional parameter in the default auto-instrumentation properties: enableForceReInstrumentation: true   apiVersion: cluster.appdynamics.com/v1alpha1 kind: Clusteragent metadata: name: k8s-cluster-agent namespace: appdynamics spec: # cluster agent properties # ... # required to enable auto-instrumentation instrumentationMethod: Env # default auto-instrumentation properties # may be overridden in an instrumentationRule containerAppCorrelationMethod: proxy nsToInstrumentRegex: default defaultAppName: "" enableForceReInstrumentation: true # ADDED # ... # one or more instrumentationRules instrumentationRules: - namespaceRegex: default customAgentConfig: -Dappdynamics.agent.reuse.nodeName=false -Dappdynamics.agent.nodeName=test # ADDED imageInfo: image: "docker.io/appdynamics/java-agent:24.8.1" agentMountPath: /opt/appdynamics imagePullPolicy: Always Afterward, please apply the changes and wait for the cluster agent to implement the new instrumentation. Then, collect the agent logs from the /opt/appdynamics-java/ver.xxx.xx/logs/ directory and attach them to the ticket. How do you collect logs from a Kubernetes pod? 1. Enter the container and pack the agent logs into a tar file. kubectl exec -it pod <pod_name> -- bash cd /opt/appdynamics-java/ver24.x.x.x/logs/ tar -cvf /java-agent-logs.tar test 2. Copy the created tar file. kubectl cp <some-namespace>/<some-pod>:/java-agent-logs.tar ./java-agent-logs.tar I hope this article was helpful/ Łukasz Kociuba ... View more