Troubleshooting AppDynamics Auto-Instrumentation Issues in Capsule Tenant

[#1]

Resolving AppDynamics Auto-Instrumentation Challenges in Capsule Tenant

This article provides a step-by-step guide to replicating a local environment using Capsule, focusing on resolving common issues with AppDynamics auto-instrumentation within Kubernetes namespaces. It covers the installation of Capsule, tenant creation, and namespace setup, followed by troubleshooting symptoms, diagnosis, and solutions. The guide is designed for technical community members seeking to optimize their Capsule and AppDynamics integration.

How to replicate locally

1. Install Capsule
   https://github.com/projectcapsule/capsule/blob/main/charts/capsule/README.md
2. Follow the instructions to create your first tenant
   https://capsule.clastix.io/docs/general/getting-started#login-as-tenant-owner
3. Create the "appdynamics" and "oil-production" namespaces; they will be attached to the same Capsule Tenant.

Symptoms

The Cluster Agent cannot auto-instrument any application.

Diagnosis

1. There are no AppDynamics annotations in the Deployment/StatefulSet description.
   
2. There is no AppDynamics annotation in the namespace description.
   
   
3. In the Cluster Agent debug logs, many errors are related to the Cluster and Capsule tenant configuration.
   A single ERROR and 'Adding instrumentation task' are repeated every 5 minutes.
   The same situation occurs in the customer environment.

[ERROR]: 2025-04-03 08:56:14 - utils.go:1086 - Unable to update namespace oil-production after 4 retries
[DEBUG]: 2025-04-03 08:56:14 - deploymenthandler.go:623 - Adding instrumentation task for deployment oil-production/java-app-to-instrument
[DEBUG]: 2025-04-03 08:56:14 - instrumentationconfig.go:794 - rule default|java|oil-production matches Deployment java-app-to-instrument in namespace oil-production with labels map[appd_agent:enable]
[DEBUG]: 2025-04-03 08:56:14 - instrumentationconfig.go:805 - Found a matching rule {default|java|oil-production .* map[appd_agent:[enable]]   java first   JAVA_TOOL_OPTIONS map[agent-mount-path:/opt/appdynamics alpine-init-container-image: image:docker.io/appdynamics/java-agent:25.2.0 image-pull-policy:Always] map[bci-enabled:true port:3892] 0 0 false false false false map[] map[]  map[] map[]    0 false []     true} for Deployment java-app-to-instrument in namespace oil-production with labels map[appd_agent:enable]
[DEBUG]: 2025-04-03 08:56:14 - instrumentationconfig.go:338 - Instrumentation state for Deployment java-app-to-instrument in namespace oil-production with labels map[appd_agent:enable] is true​

Solution

Cluster Agent service accounts require additional privileges in the Capsule Tenant:
Original Tenant

Tenant with additional privileges for Cluster Agent service accounts.

All roles available here: 
https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles

I hope this article was helpful