The Ideas platform is your space to suggest new features, vote for improvements, and collaborate with the AppDynamics community. Follow these steps to submit your ideas, vote, comment, and stay updated on enhancements that matter most to you. How to Submit a New Idea for AppDynamics Access the Ideas Portal Go to https://ideas.splunk.com/ (sign-in required). Sign In or Create a Splunk Account If you have a Splunk Account, sign in with your credentials. If not, create a new Splunk Account as prompted. Start a New Idea Submission Click Add a new Idea to begin. Enter Your Idea Details Under Choose a Workplace, select AppDynamics. Provide your Idea Title. Select the product category most relevant to your idea. (Optional) Add a Support Case number if applicable. Fill in the description field with details about your idea. Submit Your Idea Review your entry and submit. Your idea will now be visible to the community for voting and feedback. How to Vote for and Comment on Existing Ideas Access the Ideas Portal Go to https://ideas.splunk.com/ and sign in. Browse or Filter for AppDynamics Ideas Use the Filter by Category panel on the left. Expand AppDynamics to explore subcategories. Select a category to view all related ideas. Vote or Comment Vote directly from the idea list, or Click an idea to read more details and view or add comments.   How to Subscribe to Ideas for Updates Want to stay informed about an idea's progress without voting? Subscribing lets you receive updates on activity and comments. Browse and find an idea you want to follow. Click on the idea to open its details page. On the right, click the Subscribe button. You will now receive notifications about updates to this idea. Stay Involved! AppDynamics engineering teams actively monitor the Ideas portal. Your feedback shapes the future of our products-keep submitting, voting, and commenting to help us prioritize what matters to you.   More Information Check out this more detailed article to understand more on how Ideas works and how to get the most out of the platform ... View more

Building Upon AppDynamics Automation The AppDynamics Ansible Collection provides excellent automation for Java agents, Machine agents, and Database agents. To extend this automation ecosystem, I developed an Ansible solution specifically for Network Visibility (NetViz) agents that integrates seamlessly with AppDynamics’ security model. Why NetViz Automation Matters Network Visibility provides crucial insights for modern applications: 🔍 Network-level monitoring for microservices architectures 📊 Service-to-service communication mapping 🗺️ Automatic service discovery based on network flows 📈 Network performance metrics (latency, throughput, errors) 🐳 Container-aware monitoring for Docker/Kubernetes environments Manual deployment across hundreds of servers is time-consuming and error-prone. Automation is essential. The OAuth Authentication Challenge AppDynamics properly secures their agent binaries behind authentication. The key breakthrough was implementing OAuth authentication directly in Ansible to work with AppDynamics’ identity service. Here’s the authentication flow I developed: - name: Get OAuth access token uri: url: "https://identity.msrv.saas.appdynamics.com/v2.0/oauth/token" method: POST headers: Content-Type: "application/json" body_format: json body: username: "{{ vault_oauth_username }}" password: "{{ vault_oauth_password }}" scopes: ["download"] register: oauth_response no_log: true - name: Download NetViz with OAuth token get_url: url: "{{ netviz_download_url }}" dest: "/tmp/netviz-agent.deb" headers: Authorization: "Bearer {{ oauth_response.json.access_token }}" checksum: "sha256:35cb07adf9a78b2b8ffaf1c37711d1a3f362d13e1165eed52464f4b90152d2d3" Cross-Platform Architecture The solution supports both Linux and Windows environments with platform-specific optimizations: ┌─────────────────┐ ┌──────────────────┐ ┌────────────────────┐ │ Ansible │ │ Target Servers │ │ AppDynamics │ │ Control Node │ │ │ │ Controller │ │ │ │ 🐧 Linux NetViz │ │ │ │ OAuth Auth │───▶│ 🪟 Windows NetViz│───▶│ Network Metrics │ │ Cross-Platform │ │ Port 3892 │ │ Service Maps │ │ Deployment │ │ Service Mgmt │ │ Dashboards │ └─────────────────┘ └──────────────────┘ └────────────────────┘ Platform-Specific Handling The solution automatically handles platform differences: Component Linux Windows Package DEB (3.8 MB) ZIP (27.4 MB) Path /opt/appdynamics/netviz C:\AppDynamics\NetViz Service systemd Windows Service Installation apt package manager ZIP extraction + installer Step-by-Step Implementation Step 1: Repository Setup git clone https://github.com/Abhimanyu9988/appdynamics-netviz-ansible.git cd appdynamics-netviz-ansible Step 2: Configure Controller Settings Create vars/controller.yaml: # AppDynamics Controller controller_host: "your-tenant.saas.appdynamics.com" controller_port: "443" controller_ssl_enabled: "true" controller_account_name: "your-account-name" controller_access_key: "{{ vault_controller_access_key }}" # NetViz Configuration netviz_version: "25.7.0.3267" netviz_webservice_port: 3892 netviz_max_memory: "512MB" Step 3: Secure Credential Storage Use Ansible Vault for security: ansible-vault create vars/vault.yaml Add your credentials: # AppDynamics Credentials (encrypted) vault_oauth_username: "your-email@domain.com" vault_oauth_password: "your-password" vault_controller_access_key: "your-controller-access-key" Step 4: Main Deployment Playbook The core playbook (deploy-netviz.yml) handles the entire process: --- - name: Deploy AppDynamics NetViz Agent hosts: netviz_servers become: yes tasks: - name: Include vault variables include_vars: "vars/vault.yaml" - name: Get OAuth access token uri: url: "https://identity.msrv.saas.appdynamics.com/v2.0/oauth/token" method: POST body_format: json body: username: "{{ vault_oauth_username }}" password: "{{ vault_oauth_password }}" scopes: ["download"] register: oauth_response delegate_to: localhost run_once: true no_log: true - name: Download NetViz DEB get_url: url: "https://download.appdynamics.com/download/prox/download-file/netviz-deb/25.7.0.3267/appd-netviz-x64-linux-25.7.0.3267.deb" dest: "/tmp/netviz-agent.deb" headers: Authorization: "Bearer {{ oauth_response.json.access_token }}" checksum: "sha256:35cb07adf9a78b2b8ffaf1c37711d1a3f362d13e1165eed52464f4b90152d2d3" - name: Install NetViz Agent apt: deb: "/tmp/netviz-agent.deb" state: present - name: Configure NetViz Agent template: src: "templates/agent_config.lua.j2" dest: "/opt/appdynamics/netviz/conf/agent_config.lua" owner: appdynamics group: appdynamics - name: Start NetViz Service systemd: name: appd-netviz state: started enabled: yes Step 5: Configuration Template The agent configuration template ensures consistent setup: -- AppDynamics NetViz Agent Configuration agent_type = "netviz" agent_name = "{{ inventory_hostname }}_netviz_agent" -- Controller Configuration controller_host = "{{ controller_host }}" controller_port = {{ controller_port }} controller_ssl_enabled = {{ controller_ssl_enabled }} account_name = "{{ controller_account_name }}" access_key = "{{ controller_access_key }}" -- Network Configuration webservice_port = {{ netviz_webservice_port }} webservice_ip = "0.0.0.0" -- Performance Settings max_memory_usage = "{{ netviz_max_memory }}" log_level = "{{ netviz_log_level }}" Deployment in Action 🐧 Linux Deployment # Deploy to Linux localhost ansible-playbook -i inventory/localhosts.yml deploy-netviz.yaml --ask-vault-pass 🪟 Windows Deployment # Deploy to Windows localhost ansible-playbook -i inventory/windows-localhost.yml deploy-netviz-windows.yml --ask-vault-pass Cross-Platform Verification The solution includes platform-specific verification: Linux # Check service status sudo systemctl status appd-netviz # Verify network connectivity sudo netstat -tlnp | grep 3892 Windows # Check service status Get-Service "AppDynamics NetViz" # Verify network connectivity Test-NetConnection -Port 3892 Real-World Results In production environments across both Linux and Windows infrastructure, this automation has delivered: Time Savings Before: 15–20 minutes per server (manual installation) After: 2–3 minutes for entire fleet (automated deployment) Result: 90% time reduction across both platforms Consistency Before: Configuration drift between Windows and Linux environments After: Identical configuration everywhere with platform-specific optimizations Result: Zero configuration-related issues across mixed environments Security Before: Credentials shared via documentation across platforms After: Encrypted vault with unified OAuth authentication Result: Improved security posture for both Windows and Linux deployments Cross-Platform Benefits Unified workflow for mixed Windows/Linux environments Consistent OAuth authentication regardless of target platform Platform-optimized installations while maintaining configuration consistency Key Benefits Achieved ✅ One-Command Deployment: Single Ansible command deploys across entire infrastructure ✅ Secure Authentication: OAuth integration with encrypted credential storage ✅ Automatic Verification: Built-in checksum validation and service verification ✅ Production Ready: Proper user management, service configuration, and monitoring ✅ Scalable: Works for single servers or large fleet deployments Best Practices Learned Security Always use Ansible Vault for credentials Implement no_log: true for sensitive operations Verify file checksums for downloaded packages Performance Use delegate_to: localhost and run_once: true for OAuth token acquisition Configure appropriate memory limits based on server capacity Monitor resource usage in production Reliability Include comprehensive error handling Implement automatic service verification Plan for network connectivity issues Getting Started The complete solution is available on GitHub: appdynamics-netviz-ansible Quick start: git clone https://github.com/Abhimanyu9988/appdynamics-netviz-ansible.git cd appdynamics-netviz-ansible cp vars/controller.yaml.example vars/controller.yaml # Edit with your details ansible-vault create vars/vault.yaml # Add your credentials ansible-playbook -i inventory/localhost.yml deploy-netviz.yml --ask-vault-pass Conclusion This OAuth-powered, cross-platform Ansible solution transforms NetViz deployment from a manual, time-consuming process into a unified one-command operation for both Linux and Windows environments. By building upon AppDynamics’ security model and integrating with their identity service, we achieve automation, security, and platform flexibility. The solution demonstrates how to extend excellent existing automation ecosystems while respecting security boundaries, maintaining production-ready standards, and supporting diverse infrastructure environments. Network visibility is crucial for modern applications, and with proper cross-platform automation, it integrates seamlessly into your existing AppDynamics monitoring strategy regardless of your infrastructure mix. Platform Support: 🐧 Linux (Ubuntu/Debian) | 🪟 Windows (Server 2016+/Windows 10+) ... View more

Transform Your Prometheus Metrics into AppDynamics Custom Metrics in Minutes As organizations adopt cloud-native architectures, monitoring becomes increasingly complex. You might have applications exposing Prometheus metrics, but your enterprise standardizes on AppDynamics for observability, alerting, and business dashboards. What if you could monitor ANY Prometheus-enabled application in AppDynamics without changing a single line of code? In this article, I’ll show you how to build a Universal Prometheus to AppDynamics converter that automatically transforms any Prometheus metrics into AppDynamics Custom Metrics, complete with organized hierarchies and human-readable names. The Challenge: Two Monitoring Worlds The Prometheus World Cloud-native standard for metrics Pull-based model with time-series data Flexible labeling system Developer-friendly metric naming (snake_case) The AppDynamics World Enterprise monitoring platform Business context and correlation Advanced alerting and dashboards Hierarchical metric organization The gap? No native way to import Prometheus metrics into AppDynamics while preserving the rich label context and maintaining organized metric hierarchies. The Solution: Universal Converter Our solution uses AppDynamics Machine Agent Custom Extensions to: ✅ Automatically discover all Prometheus metrics ✅ Convert technical names to business-friendly format ✅ Preserve label context as hierarchical paths ✅ Zero application changes required ✅ Works with ANY Prometheus endpoint Architecture Overview ┌─────────────────┐ ┌──────────────────┐ ┌────────────────────┐ │ Application │ │ Machine Agent │ │ AppDynamics │ │ │ │ │ │ Controller │ │ Prometheus │───▶│ Universal │───▶│ │ │ /metrics │ │ Converter │ │ Custom Metrics │ │ Endpoint │ │ Extension │ │ Dashboards │ └─────────────────┘ └──────────────────┘ └────────────────────┘ Data Flow: Application exposes metrics at /actuator/prometheus Machine Agent extension scrapes metrics every 60 seconds Universal converter transforms format and creates hierarchy AppDynamics Controller receives organized custom metrics Business users create dashboards and alerts Step 1: Clone Repository and Start Sample Application Our sample application is a comprehensive e-commerce microservice that demonstrates realistic Prometheus metrics for AppDynamics integration. Install and Run # Clone the repository git clone https://github.com/Abhimanyu9988/universal-prometheus-appdynamics cd universal-prometheus-appdynamics # Install dependencies pip install -r requirements.txt # Start the sample application python sample-app/microservice.py Application will start with: Metrics endpoint: http://localhost:8080/actuator/prometheus Health check: http://localhost:8080/health Order API: POST http://localhost:8080/api/orders User sessions: POST http://localhost:8080/api/users/<id>/session Cart abandonment: POST http://localhost:8080/api/cart/abandon Complete E-commerce Sample Application Our sample application simulates a realistic e-commerce microservice with comprehensive business and infrastructure metrics: Key Features: Business Metrics: Orders, revenue, user activity, conversion rates Infrastructure Metrics: Database connections, cache performance, memory/CPU usage API Endpoints: Order processing, user sessions, cart abandonment tracking Background Simulation: Realistic metric generation with temporal patterns Quick Start: # Clone the repository git clone https://github.com/Abhimanyu9988/universal-prometheus-appdynamics cd universal-prometheus-appdynamics # Install dependencies pip install -r requirements.txt # Start the sample application python sample-app/microservice.py Sample Metrics Generated: Curl below http://localhost:8080/actuator/prometheus to see metrics like: # Business Metrics orders_processed_total{product_category="electronics",region="us_east",payment_method="credit_card"} 15 order_value_dollars{product_category="electronics",customer_tier="gold"} 750 active_users_current{user_type="premium",subscription_tier="pro"} 45 conversion_rate_percentage{traffic_source="organic",product_category="electronics"} 6.2 # Infrastructure Metrics database_connections_active{database_name="user_db",connection_type="read"} 12 cache_hits_total{cache_type="redis",cache_key_pattern="user_*"} 1247 memory_usage_bytes{memory_type="heap"} 856000000 api_requests_total{method="POST",endpoint="/api/orders",status_code="200"} 89 Generate Test Data: # Create orders curl -X POST http://localhost:8080/api/orders curl -X POST http://localhost:8080/api/orders # User sessions curl -X POST http://localhost:8080/api/users/user123/session # Cart abandonment curl -X POST http://localhost:8080/api/cart/abandon The complete application code is available in the GitHub repository at sample-app/microservice.py with additional features including: Comprehensive metric types: Counters, gauges, histograms, and summaries Realistic business scenarios: E-commerce order processing, user activity, payment methods Infrastructure monitoring: Database connections, cache performance, memory/CPU usage Background simulation: Automatic metric generation with realistic patterns API endpoints: Interactive endpoints to generate test data Production patterns: Proper error handling, logging, and metric organization 📁 Full Implementation: View on GitHub Run the Application python3 sample_microservice.py Visit http://localhost:8080/actuator/prometheus to see metrics like: # HELP orders_processed_total Total orders processed # TYPE orders_processed_total counter orders_processed_total{product_type="electronics",region="us-east"} 15.0 orders_processed_total{product_type="clothing",region="europe"} 8.0 # HELP active_users_current Current number of active users # TYPE active_users_current gauge active_users_current{user_type="premium"} 45.0 active_users_current{user_type="standard"} 78.0 Step 2: Universal Prometheus to AppDynamics Converter Now let’s create the Universal Converter that transforms ANY Prometheus metrics into AppDynamics format: The Magic: Conversion Logic The Universal Converter transforms any Prometheus metrics into AppDynamics format. Here’s how it works with our sample application: Input (Prometheus Format): orders_processed_total{product_category="electronics",region="us_east",payment_method="credit_card"} 15 active_users_current{user_type="premium",subscription_tier="pro"} 45 database_connections_active{database_name="user_db",connection_type="read"} 12 Output (AppDynamics Format): name=Custom Metrics|App|Orders Processed Total|Product Category Electronics|Region Us East|Payment Method Credit Card,value=15,aggregator=OBSERVATION,time-rollup=CURRENT,cluster-rollup=COLLECTIVE name=Custom Metrics|App|Active Users Current|User Type Premium|Subscription Tier Pro,value=45,aggregator=OBSERVATION,time-rollup=CURRENT,cluster-rollup=INDIVIDUAL name=Custom Metrics|App|Database Connections Active|Database Name User Db|Connection Type Read,value=12,aggregator=OBSERVATION,time-rollup=CURRENT,cluster-rollup=INDIVIDUAL Core Conversion Functions: Complete code : #!/bin/bash # Universal Prometheus to AppDynamics Converter # Transforms: orders_processed_total{product_type="electronics"} 15 # Into: Custom Metrics|App|Orders Processed Total|Product Type Electronics,value=15 PROMETHEUS_ENDPOINT="http://localhost:8080/actuator/prometheus" MAX_METRICS=100 # Convert snake_case to Title Case convert_to_title_case() { echo "$1" | sed 's/_/ /g' | sed 's/\b\w/\U&/g' } # Clean label values for AppDynamics hierarchy clean_label_value() { echo "$1" | sed 's/"//g' | sed 's/[^a-zA-Z0-9.]/ /g' | sed 's/\b\w/\U&/g' } # Process each metric line process_metric_line() { local line="$1" # Parse: metric_name{labels} value if [[ "$line" =~ ^([a-zA-Z_][a-zA-Z0-9_]*)(.*)[[:space:]]+([0-9.]+)$ ]]; then local metric_name="${BASH_REMATCH[1]}" local labels_part="${BASH_REMATCH[2]}" local value="${BASH_REMATCH[3]}" # Convert to human readable local readable_name=$(convert_to_title_case "$metric_name") local metric_path="Custom Metrics|App|$readable_name" # Process labels into hierarchy if [[ "$labels_part" =~ ^\{.*\}$ ]]; then labels_part="${labels_part#\{}" labels_part="${labels_part%\}}" IFS=',' read -ra LABELS <<< "$labels_part" for label in "${LABELS[@]}"; do if [[ "$label" =~ ^[[:space:]]*([^=]+)=[[:space:]]*\"?([^\"]+)\"?$ ]]; then local key=$(convert_to_title_case "${BASH_REMATCH[1]}") local val=$(clean_label_value "${BASH_REMATCH[2]}") metric_path="$metric_path|$key $val" fi done fi # Output AppDynamics format echo "name=$metric_path,value=${value%.*},aggregator=OBSERVATION,time-rollup=CURRENT,cluster-rollup=INDIVIDUAL" fi } # Main execution METRICS_DATA=$(curl -s --max-time 30 "$PROMETHEUS_ENDPOINT") if [ $? -ne 0 ]; then echo "name=Custom Metrics|App|Connection Status,value=0,aggregator=OBSERVATION,time-rollup=CURRENT,cluster-rollup=INDIVIDUAL" exit 1 fi echo "name=Custom Metrics|App|Connection Status,value=1,aggregator=OBSERVATION,time-rollup=CURRENT,cluster-rollup=INDIVIDUAL" # Process all metrics metric_count=0 while IFS= read -r line && [ $metric_count -lt $MAX_METRICS ]; do result=$(process_metric_line "$line") if [ ! -z "$result" ]; then echo "$result" ((metric_count++)) fi done <<< "$METRICS_DATA" What This Creates in AppDynamics Before (Prometheus): orders_processed_total{product_category="electronics",region="us_east",payment_method="credit_card"} 15 active_users_current{user_type="premium",subscription_tier="pro"} 45 database_connections_active{database_name="user_db",connection_type="read"} 12 After (AppDynamics Hierarchy): Custom Metrics └── App ├── Orders Processed Total │ └── Product Category Electronics │ └── Region Us East │ └── Payment Method Credit Card = 15 ├── Active Users Current │ └── User Type Premium │ └── Subscription Tier Pro = 45 └── Database Connections Active └── Database Name User Db └── Connection Type Read = 12 Step 3: Deploy to AppDynamics Machine Agent Create Extension Directory mkdir -p /opt/appdynamics/machine-agent/monitors/UniversalPrometheusMonitor cd /opt/appdynamics/machine-agent/monitors/UniversalPrometheusMonitor Save the Converter Script # Save the full converter script cat > UniversalPrometheusMonitor.sh << 'EOF' [Full script content from above] EOF chmod +x UniversalPrometheusMonitor.sh Create monitor.xml <?xml version="1.0" encoding="UTF-8"?> <monitor> <name>UniversalPrometheusMonitor</name> <type>managed</type> <description>Universal Prometheus to AppDynamics Converter</description> <monitor-run-task> <execution-style>periodic</execution-style> <execution-frequency-in-seconds>60</execution-frequency-in-seconds> <name>Universal Prometheus Collection</name> <type>executable</type> <execution-timeout-in-secs>60</execution-timeout-in-secs> <task-arguments> </task-arguments> <executable-task> <type>file</type> <file>UniversalPrometheusMonitor.sh</file> </executable-task> </monitor-run-task> </monitor> Test and Deploy # Test manually ./UniversalPrometheusMonitor.sh # Restart Machine Agent sudo systemctl restart machine-agent Step 4: Verify in AppDynamics Controller After 2–3 minutes, navigate to: Metric Browser → Application Infrastructure Performance → [Machine Agent] → Custom Metrics → App You’ll see organized metrics like: Orders Processed Total Product Type Electronics → Region Us East Product Type Clothing → Region Europe Active Users Current User Type Premium User Type Standard Database Connections Active Database Name User Db Database Name Order Db   Real-World Applications This solution works perfectly for: 🏗️ Cloud-Native Applications Kubernetes deployments with Prometheus metrics Microservices exposing custom business metrics Container workloads with infrastructure metrics 🔧 Enterprise Integration Camunda workflow engines (both Platform 7 & 😎 Apache Kafka clusters Elasticsearch deployments Custom applications with Prometheus client libraries 📊 Business Intelligence E-commerce metrics (orders, revenue, cart abandonment) SLA monitoring (response times, error rates) Capacity planning (resource utilization, growth trends) Advanced Configuration Customize Metric Filtering # Filter specific metrics INCLUDE_PATTERNS="orders_|users_|database_" EXCLUDE_PATTERNS="_bucket|_sum|_count" if [[ "$metric_name" =~ $INCLUDE_PATTERNS ]] && [[ ! "$metric_name" =~ $EXCLUDE_PATTERNS ]]; then # Process metric fi Add Business Context # Map technical names to business terms case "$metric_name" in "orders_processed_total") readable_name="Revenue Orders";; "active_users_current") readable_name="Online Customers";; "database_connections_active") readable_name="DB Performance";; *) readable_name=$(convert_to_title_case "$metric_name");; esac Configure Aggregation Types # Set appropriate aggregation based on metric type if [[ "$metric_name" =~ _total$ ]]; then aggregator="OBSERVATION" cluster_rollup="COLLECTIVE" elif [[ "$metric_name" =~ _current$ ]]; then aggregator="OBSERVATION" cluster_rollup="INDIVIDUAL" fi Monitoring and Troubleshooting Health Checks The converter includes built-in monitoring: # Connection status name=Custom Metrics|App|Connection Status,value=1 # Metrics processed name=Custom Metrics|App|Monitoring|Total Metrics Processed,value=67 Debugging # Test the converter manually ./UniversalPrometheusMonitor.sh | head -20 # Check Machine Agent logs tail -f /opt/appdynamics/machine-agent/logs/machine-agent.log | grep Universal # Verify endpoint connectivity curl http://localhost:8080/actuator/prometheus | head -10 Performance Considerations Metric Volume Management MAX_METRICS=100 # Prevent overwhelming AppDynamics BATCH_SIZE=50 # Process in batches for large metric sets Memory Usage # For high-volume environments TIMEOUT=10 # Shorter timeouts FREQUENCY=120 # Less frequent collection (2 minutes) Network Optimization # Use local endpoints when possible PROMETHEUS_ENDPOINT="http://localhost:8080/actuator/prometheus" # Add retry logic for i in {1..3}; do METRICS_DATA=$(curl -s --max-time $TIMEOUT "$PROMETHEUS_ENDPOINT") [ $? -eq 0 ] && break sleep 5 done Security Best Practices Authentication # For secured Prometheus endpoints curl -H "Authorization: Bearer $TOKEN" "$PROMETHEUS_ENDPOINT" # Or with basic auth curl -u "$USERNAME:$PASSWORD" "$PROMETHEUS_ENDPOINT" SSL/TLS # For HTTPS endpoints curl --cacert /path/to/ca.crt "$PROMETHEUS_ENDPOINT" Network Security ✅ Restrict access to Prometheus endpoints ✅ Use internal networks for metric collection ✅ Monitor extension logs for security events ✅ Rotate credentials regularly Scaling Across Environments Multi-Environment Setup # Development PROMETHEUS_ENDPOINT="http://dev-app:8080/actuator/prometheus" METRIC_PREFIX="Custom Metrics|Dev|App" # Production PROMETHEUS_ENDPOINT="http://prod-app:8080/actuator/prometheus" METRIC_PREFIX="Custom Metrics|Prod|App" Multiple Applications Create separate extensions for each application: monitors/ ├── App1PrometheusMonitor/ ├── App2PrometheusMonitor/ └── CamundaPrometheusMonitor/ Conclusion This Universal Prometheus to AppDynamics converter bridges the gap between cloud-native monitoring and enterprise observability platforms. Key Benefits: ✅ Zero code changes to existing applications ✅ Automatic metric discovery and organization ✅ Human-readable hierarchies in AppDynamics ✅ Enterprise-grade monitoring with business context ✅ Works with ANY Prometheus endpoint Perfect For: DevOps teams adopting cloud-native technologies Enterprise architects standardizing on AppDynamics SRE engineers needing comprehensive observability Business stakeholders requiring custom dashboards The solution is production-ready, scalable, and maintainable — giving you the best of both worlds: Prometheus’s flexibility with AppDynamics’s enterprise capabilities. Resources 📁 GitHub Repository: universal-prometheus-appdynamics 📖 AppDynamics Documentation: Machine Agent Extensions 🔧 Prometheus Client Libraries: prometheus.io/docs/instrumenting/clientlibs Found this helpful? ⭐ Star the repository and share your success stories in the comments below! Have questions? 💬 Join the discussion and let’s build better monitoring solutions together. This article is part of a series on enterprise observability and monitoring best practices. Follow for more practical guides on AppDynamics, Prometheus, and modern monitoring architectures. ... View more

Building a production-ready monitoring solution for GCP Pub/Sub that provides comprehensive visibility into your messaging infrastructure   The Challenge Google Cloud Pub/Sub is the backbone of many modern distributed systems, handling millions of messages daily. But here’s the problem: how do you monitor something you can’t see? Most organizations struggle with: ❌ Limited visibility into topic and subscription health ❌ Reactive monitoring — finding out about issues after they impact users ❌ Fragmented metrics across different tools and dashboards ❌ Complex setup requiring deep GCP and monitoring expertise What if I told you there’s a way to get comprehensive Pub/Sub monitoring integrated directly into your existing AppDynamics dashboard with minimal setup? The Solution: End-to-End Automation In this guide, we’ll build a complete monitoring solution that: ✅ Automatically creates GCP Pub/Sub topics and subscriptions ✅ Generates realistic test data for immediate monitoring ✅ Collects 50+ metrics covering every aspect of your Pub/Sub infrastructure ✅ Integrates seamlessly with AppDynamics Machine Agent ✅ Supports multiple platforms (AWS, GCP, Azure, on-premises) ✅ Includes cleanup scripts for easy environment management Architecture Overview Here’s what we’re building: ┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │ GCP Pub/Sub │ │ Metrics Script │ │ AppDynamics │ │ │ │ │ │ │ │ ┌─────────────┐ │ │ ┌──────────────┐ │ │ ┌─────────────┐ │ │ │ Topics │◄┼────┼─│ gcloud │ │ │ │ Machine │ │ │ └─────────────┘ │ │ │ CLI │ │ │ │ Agent │ │ │ │ │ └──────────────┘ │ │ └─────────────┘ │ │ ┌─────────────┐ │ │ ┌──────────────┐ │ │ │ │ │ │Subscriptions│◄┼────┼─│ Metrics │─┼────┼────────┘ │ │ └─────────────┘ │ │ │ Collector │ │ │ │ │ │ │ └──────────────┘ │ │ ┌─────────────┐ │ │ ┌─────────────┐ │ │ ┌──────────────┐ │ │ │ Controller │ │ │ │ Metrics │◄┼────┼─│ JSON │ │ │ │ Dashboard │ │ │ └─────────────┘ │ │ │ Auth │ │ │ └─────────────┘ │ └─────────────────┘ │ └──────────────┘ │ └─────────────────┘ └──────────────────┘ Prerequisites Before we start, ensure you have: GCP Project with billing enabled AppDynamics Machine Agent installed and running Linux system (we support Amazon Linux 2/2023, Ubuntu, RHEL/CentOS, Rocky Linux) Basic command-line knowledge Don’t worry if you’re missing some pieces — we’ll guide you through everything! Part 1: Setting Up the Foundation 1.1 Clone the Repository First, let’s get our toolkit: git clone https://github.com/Abhimanyu9988/gcp-pubsub-appdynamics.git cd gcp-pubsub-appdynamics chmod +x *.sh 1.2 Install Prerequisites Our smart installer detects your operating system and installs everything needed: bash sudo ./ec2-pre-req.sh What this does: 🔧 Detects your OS (Amazon Linux 2/2023, Ubuntu, RHEL, etc.) 📥 Installs Google Cloud SDK with proper Python version 🛠️ Installs required tools (jq, curl, etc.) ⚙️ Sets up proper permissions and paths ✅ Validates everything works Sample output: ============================================== Multi-Distribution GCP Pub/Sub Prerequisites ============================================== [INFO] Detected OS: Amazon Linux 2 [SUCCESS] System packages updated [SUCCESS] Python 3.9 installed [SUCCESS] ✅ Google Cloud SDK installed successfully [SUCCESS] 🎉 All prerequisites installed successfully! Part 2: GCP Service Account Setup 2.1 Create Service Account (One-time Setup) On your local machine where you have gcloud configured: bash ./create_service_account.sh What this creates: 🔐 Service account with minimal required permissions 🗝️ JSON key file for authentication 🛡️ IAM roles: pubsub.viewer, monitoring.viewer, serviceusage.serviceUsageViewer Sample output: bash ============================================ GCP Service Account Creation for Pub/Sub Monitoring ============================================ [SUCCESS] Using project: my-gcp-project [SUCCESS] Created service account: pubsub-monitor@my-project.iam.gserviceaccount.com [SUCCESS] ✅ Assigned: roles/pubsub.viewer [SUCCESS] ✅ Service account key created: pubsub-monitor-service-account.json 📦 Transfer to AWS Linux 2: scp -i your-key.pem pubsub-monitor-service-account.json ec2-user@your-instance-ip:~/ 2.2 Transfer Credentials Copy the service account key to your monitoring server: bash # On your local machine scp -i your-aws-key.pem pubsub-monitor-service-account.json ec2-user@your-server:~/ # On your monitoring server sudo mkdir -p /opt/appdynamics sudo mv ~/pubsub-monitor-service-account.json /opt/appdynamics/ sudo chmod 600 /opt/appdynamics/pubsub-monitor-service-account.json Part 3: Creating Pub/Sub Resources 3.1 Set Up Your Environment bash export GCP_PROJECT_ID="your-actual-project-id" 3.2 Create Topics and Subscriptions bash ./pubsub_create.sh What this does: 🎯 Creates Pub/Sub topic: appdynamics-monitoring-topic 📥 Creates subscription: appdynamics-monitoring-subscription 📊 Publishes 100 sample messages with realistic data 📈 Generates custom metrics for immediate monitoring 🔄 Optional: Runs continuous simulation workload Sample output: bash ================================== GCP Pub/Sub Creation Script ================================== [SUCCESS] Created topic: appdynamics-monitoring-topic [SUCCESS] Created subscription: appdynamics-monitoring-subscription [INFO] Publishing 100 messages to appdynamics-monitoring-topic [SUCCESS] Publishing completed: Total messages: 100 Published successfully: 100 Failed to publish: 0 Messages per second: 23 3.3 Verify Resources bash ./pubsub_info.sh status This shows you exactly what was created and provides troubleshooting information. Part 4: Configure Metrics Collection 4.1 Edit the Main Script Update the configuration in script.sh: bash vi script.sh # Update these values: PROJECT_ID="your-actual-project-id" SERVICE_ACCOUNT_KEY_FILE="/opt/appdynamics/pubsub-monitor-service-account.json" # Optional: Add more topics/subscriptions TOPIC_NAMES="topic1,topic2,topic3" SUBSCRIPTION_NAMES="sub1,sub2,sub3" 4.2 Test Metrics Collection bash ./script.sh Expected output (50+ metrics): bash name=Custom Metrics|PubSub|Health|Collection Success, value=1 name=Custom Metrics|PubSub|Topic|appdynamics-monitoring-topic|Status, value=1 name=Custom Metrics|PubSub|Subscription|appdynamics-monitoring-subscription|Ack Deadline, value=60 name=Custom Metrics|PubSub|API|PubSub Enabled, value=1 name=Custom Metrics|PubSub|Project|Total Topics, value=5 Part 5: AppDynamics Integration 5.1 Install Machine Agent Extension bash # Create extension directory sudo mkdir -p /opt/appdynamics/machine-agent/monitors/PubSubMonitor # Copy files sudo cp script.sh /opt/appdynamics/machine-agent/monitors/PubSubMonitor/ sudo cp monitor.xml /opt/appdynamics/machine-agent/monitors/PubSubMonitor/ # Set permissions sudo chown -R appdynamics:appdynamics /opt/appdynamics/machine-agent/monitors/PubSubMonitor sudo chmod +x /opt/appdynamics/machine-agent/monitors/PubSubMonitor/script.sh 5.2 Restart Machine Agent bash sudo systemctl restart appdynamics-machine-agent # Verify it's running sudo systemctl status appdynamics-machine-agent 5.3 Check Logs bash # Check Machine Agent logs tail -f /opt/appdynamics/machine-agent/logs/machine-agent.log # Check for our extension grep -i pubsub /opt/appdynamics/machine-agent/logs/machine-agent.log Part 6: The Metrics Deep Dive Our solution collects 50+ comprehensive metrics across these categories: 🎯 Topic Metrics Status & Health: Topic accessibility, configuration validation Subscription Management: Count and health of attached subscriptions Security: IAM policy accessibility Configuration: Message retention settings 📥 Subscription Metrics Operational Health: Status, acknowledgment deadlines Configuration Analysis: Push vs Pull detection, retry policies Advanced Features: Dead letter queues, message filtering Performance: Message retention, delivery settings 🔌 API & Service Health Service Availability: Pub/Sub and Monitoring API status Operational Capabilities: List topics/subscriptions permissions Health Scoring: Overall API accessibility 📊 Project-Level Insights Resource Inventory: Total topics and subscriptions across project Configuration Validation: Setup verification Custom Metrics Integration: Deployment script metrics Part 7: Viewing in AppDynamics 7.1 Navigate to Metrics Log into your AppDynamics Controller Go to Servers → Machine Agents → Your Server Navigate to Your Server → Custom Metrics Look for Custom Metrics | PubSub 7.2 Create Dashboards Create custom dashboards with widgets for: Health Overview: Collection Success Rate GCP Connectivity Status API Health Score Topic Performance: Topic Status by Name Subscription Count Distribution Success Rate Trends Subscription Analytics: Ack Deadline Distribution Push vs Pull Breakdown Dead Letter Queue Usage Project Insights: Total Resource Counts Configuration Compliance Custom Metrics Trends Part 8: Production Deployment 8.1 Security Best Practices bash # Secure service account keys sudo chmod 600 /opt/appdynamics/*.json sudo chown appdynamics:appdynamics /opt/appdynamics/*.json # Enable audit logging gcloud logging sinks create pubsub-monitoring-sink \ bigquery.googleapis.com/projects/YOUR_PROJECT/datasets/audit_logs \ --log-filter='protoPayload.serviceName="pubsub.googleapis.com"' 8.2 Monitoring Multiple Environments bash # Development environment export GCP_PROJECT_ID="dev-project" export TOPIC_NAMES="dev-orders,dev-inventory" # Production environment export GCP_PROJECT_ID="prod-project" export TOPIC_NAMES="orders,inventory,notifications,analytics" 8.3 Alerting Setup Create AppDynamics policies for: Topic/Subscription availability < 100% Collection errors > 0 API health score < 1 Custom metric age > 5 minutes Part 9: Maintenance & Operations 9.1 Resource Management bash # Check current resources ./pubsub_info.sh status # View available metrics ./pubsub_info.sh metrics # Generate more test data RUN_SIMULATION=true ./pubsub_create.sh # Clean up everything ./pubsub_destroy.sh 9.2 Service Account Rotation bash # Create new service account ./create_service_account.sh # Test with new credentials ./script.sh # Delete old service account ./delete_service_account.sh 9.3 Troubleshooting Common Issues & Solutions: Authentication Errors: bash # Verify service account file jq empty /opt/appdynamics/pubsub-monitor-service-account.json # Test authentication gcloud auth activate-service-account --key-file=/opt/appdynamics/pubsub-monitor-service-account.json Permission Denied: bash # Check IAM roles gcloud projects get-iam-policy YOUR_PROJECT_ID \ --flatten="bindings[].members" \ --filter="bindings.members:your-service-account@project.iam.gserviceaccount.com" No Metrics in AppDynamics: bash # Test script manually cd /opt/appdynamics/machine-agent/monitors/PubSubMonitor ./script.sh # Check Machine Agent logs tail -f /opt/appdynamics/machine-agent/logs/machine-agent.log Advanced Use Cases Multi-Project Monitoring Monitor Pub/Sub across multiple GCP projects by deploying separate instances with different service accounts: bash # Project 1 - Development PROJECT_ID="dev-project" SERVICE_ACCOUNT_KEY_FILE="/opt/appdynamics/dev-sa.json" ./script.sh # Project 2 - Production PROJECT_ID="prod-project" SERVICE_ACCOUNT_KEY_FILE="/opt/appdynamics/prod-sa.json" ./script.sh Custom Metrics Integration Extend the solution to include your application-specific metrics: bash # Add custom metrics to /tmp/pubsub_custom_metrics.log echo "[$(date -u +"%Y-%m-%dT%H:%M:%S.%3NZ")] CUSTOM_METRIC order_processing_rate 150" >> /tmp/pubsub_custom_metrics.log CI/CD Integration Include in your deployment pipelines: yaml # GitHub Actions example - name: Setup Pub/Sub Monitoring run: | ./pubsub_create.sh ./script.sh # Verify metrics collection if grep -q "Collection Success, value=1" <(./script.sh); then echo "✅ Monitoring setup successful" else echo "❌ Monitoring setup failed" exit 1 fi Performance & Scaling Our solution is designed for production scale: ⚡ Fast Execution: Collects 50+ metrics in under 10 seconds 🔄 Concurrent Safe: Multiple instances can run simultaneously 📊 Scalable: Handles hundreds of topics/subscriptions 💾 Lightweight: Minimal memory and CPU footprint 🛡️ Resilient: Continues partial collection even if some resources fail Performance Metrics: Average collection time: 5-7 seconds Memory usage: < 50MB during collection Network requests: Optimized with bulk operations Error recovery: Continues on partial failures What's Next? You now have a production-ready Pub/Sub monitoring solution that provides: ✅ Complete Visibility - 50+ metrics covering every aspect ✅ Automated Setup - One-click deployment and configuration ✅ Integration Ready - Native AppDynamics dashboard integration ✅ Production Tested - Handles real-world scale and complexity ✅ Maintenance Friendly - Easy updates, rotation, and cleanup Take It Further 📊 Create Custom Dashboards with business-specific KPIs 🚨 Set Up Intelligent Alerting for proactive issue detection 🔄 Integrate with CI/CD for automated environment monitoring 📈 Add Business Metrics specific to your use cases 🌐 Scale to Multiple Projects and regions Resources & Links 📚 Full Documentation: GitHub Repository 🛠️ Issues & Support: GitHub Issues 📖 GCP Pub/Sub Docs: Official Documentation 🎯 AppDynamics Extensions: Machine Agent Extensions Conclusion Monitoring GCP Pub/Sub doesn't have to be complicated. With the right automation and tooling, you can have comprehensive visibility into your messaging infrastructure in under 30 minutes. The solution we've built together provides enterprise-grade monitoring with minimal operational overhead. Your teams can now: Detect issues proactively before they impact users Understand performance patterns across your messaging layer Make data-driven decisions about scaling and optimization Maintain high availability with real-time health monitoring Have questions or want to share your success story? Drop a comment below or reach out on GitHub! ⭐ If this guide helped you, please give the GitHub repository a star and share it with your team! ... View more

There will be times when multiple teams in your Organization will want to monitor and instrument their Applications. They would also want to have these applications report to different AppDynamics Backend (Controller) The latest Cluster agent releases gives you that option. Below has been tested on Cluster Agent 25.5 release.  First create 2 Applications: 1. tomcat-sample.yaml --- apiVersion: apps/v1 kind: Deployment metadata: name: tomcat-app labels: app: tomcat-app-java-apps namespace: java-apps spec: replicas: 1 selector: matchLabels: app: tomcat-app template: metadata: labels: app: tomcat-app spec: containers: - name: tomcat-app #image: docker.io/abhimanyubajaj98/tomcat-sample:latest image: docker.io/abhimanyubajaj98/tomcat-sample imagePullPolicy: Always ports: - containerPort: 8080 env: - name: JAVA_TOOL_OPTIONS value: "-Xmx512m" --- apiVersion: v1 kind: Service metadata: name: tomcat-app-service labels: app: tomcat-app namespace: java-apps spec: ports: - port: 8080 targetPort: 8080 selector: app: tomcat-app   2. tomcat-sample-ces.yaml --- apiVersion: apps/v1 kind: Deployment metadata: name: tomcat-app-1 labels: app: tomcat-app-java-apps-1 namespace: one-apps spec: replicas: 1 selector: matchLabels: app: tomcat-app-1 template: metadata: labels: app: tomcat-app-1 spec: containers: - name: tomcat-app-1 #image: docker.io/abhimanyubajaj98/tomcat-sample:latest image: docker.io/abhimanyubajaj98/tomcat-sample imagePullPolicy: Always ports: - containerPort: 8080 env: - name: JAVA_TOOL_OPTIONS value: "-Xmx512m" --- apiVersion: v1 kind: Service metadata: name: tomcat-app-service-1 labels: app: tomcat-app-1 namespace: one-apps spec: ports: - port: 8080 targetPort: 8080 selector: app: tomcat-app-1   To deploy-> 1. kubectl create ns java-apps one-apps 2. kubectl create -f tomcat-sample.yaml 3. kubectl create -f tomcat-sample-ces.yaml   Once deployed: root@ip-172-31-12-116:/opt/appdynamics/java-apps# kubectl -n java-apps get pods NAME READY STATUS RESTARTS AGE tomcat-app-76df69dc7b-f74wv 1/1 Running 0 71m root@ip-172-31-12-116:/opt/appdynamics/java-apps# kubectl -n one-apps get pods NAME READY STATUS RESTARTS AGE tomcat-app-1-6b86c4b444-h65sj 1/1 Running 0 20m   Auto-instrument For this I will deploy one cluster agent with Helm charts and the other one with normal kubectl command line. First up is creating the secret. Remember Cluster agent will be deployed in 2 namespaces so we will need to create 2 secrets in different namespace.  Our 1st namespace  : appdynamics Our 2nd namespace : appdynamics-1 Let's create secret: kubectl -n appdynamics create secret generic cluster-agent-secret --from-literal=controller-key=<access-key> kubectl -n appdynamics-1 create secret generic cluster-agent-secret --from-literal=controller-key=<access-key>   First Cluster Agent deployment in appdynamics namespace My helm: root@ip-172-31-12-116:/opt/appdynamics/cluster-agent/cluster-agent-alpine-arm64-bundled-distribution/helm-charts# cat values.yaml installClusterAgent: true installInfraViz: false installSplunkOtelCollector: false imageInfo: agentImage: docker.io/appdynamics/cluster-agent agentTag: 25.5.0-1126 operatorImage: docker.io/appdynamics/cluster-agent-operator operatorTag: 25.5.0-1107 imagePullPolicy: Always machineAgentImage: docker.io/appdynamics/machine-agent machineAgentTag: latest machineAgentWinImage: docker.io/appdynamics/machine-agent-analytics machineAgentWinTag: win-latest netVizImage: docker.io/appdynamics/machine-agent-netviz netvizTag: latest controllerInfo: url: https://controllerces.saas.appdynamics.com:443 account: controllerces username: null password: null accessKey: null globalAccount: controllerxxx_xxxxxx3 customSSLCert: null keyStorePasswordSecret: '' keyStoreFileSecret: '' authenticateProxy: false proxyUrl: null proxyUser: null proxyPassword: null createServiceAccount: true clusterAgent: containerProperties: containerBatchSize: 5 containerParallelRequestLimit: 1 containerRegistrationInterval: 120 logProperties: logFileSizeMb: 5 logFileBackups: 3 logLevel: DEBUG metricProperties: metricsSyncInterval: 30 metricUploadRetryCount: 2 metricUploadRetryIntervalMilliSeconds: 5 podMetricCollectionMaxGoRoutines: 3 podMetricCollectionRequestTimeoutSeconds: 5 nsToMonitorRegex: .* appName: two-ca instrumentationConfig: enabled: true containerAppCorrelationMethod: proxy instrumentationMethod: Env numberOfTaskWorkers: 5 appNameStrategy: label nsToInstrumentRegex: java-apps|nodejs-apps|dotnet-apps instrumentationRules: - namespaceRegex: java-apps language: java appNameLabel: app runAsUser: 999 runAsGroup: 999 imageInfo: image: docker.io/appdynamics/java-agent:latest agentMountPath: /opt/appdynamics imagePullPolicy: Always   helm install -f values.yaml cluster-agent -n appdynamcis  root@ip-172-31-12-116:/opt/appdynamics/cluster-agent/cluster-agent-alpine-arm64-bundled-distribution/helm-charts# kubectl -n appdynamics get pods NAME READY STATUS RESTARTS AGE appdynamics-operator-769dcf8f4b-kjm2z 1/1 Running 0 77m two-ca-appdynamics-54bc66f55c-5qzfk 1/1 Running 0 77m   Second Cluster Agent deployment in appdynamics namespace Before we deploy the second Cluster Agent, we will need to edit our cluster-agent-operator.yaml file. What we need to make sure is every place where you have  namespace: appdynamics This gets changed to namespace: appdynamics-1   I have uploaded cluster-agent-operator.txt file that has this configuration. You can use with 25.5 version of Cluster Agent by changing the extension to yaml Do-> kubectl create -f cluster-agent-operator.yaml My cluster-agent.yaml: root@ip-172-31-12-116:/opt/appdynamics/cluster-agent/cluster-agent-alpine-arm64-bundled-distribution# cat cluster-agent.yaml apiVersion: cluster.appdynamics.com/v1alpha1 kind: Clusteragent metadata: name: k8s-cluster-agent namespace: appdynamics-1 spec: appName: "cluster-2" controllerUrl: "https://ces-controller.saas.appdynamics.com:443" account: "ces-controller" # docker image info image: "docker.io/appdynamics/cluster-agent:latest" serviceAccountName: appdynamics-cluster-agent nsToMonitorRegex: appdynamics resources: limits: cpu: 90m memory: "200Mi" requests: cpu: 90m memory: "100Mi" instrumentationMethod: Env nsToInstrumentRegex: one-apps appNameStrategy: label instrumentationRules: - namespaceRegex: one-apps language: java appNameLabel: app runAsUser: 999 runAsGroup: 999 imageInfo: image: docker.io/appdynamics/java-agent:latest agentMountPath: /opt/appdynamics imagePullPolicy: Always ### Uncomment the following line if you need pull secret #imagePullSecret: "<your-docker-pull-secret-name>" ### Uncomment the following line if you need to enable the profiling #pprofEnabled: true #pprofPort: 9991   kubectl create -f cluster-agent.yaml Once done: root@ip-172-31-12-116:/opt/appdynamics/cluster-agent/cluster-agent-alpine-arm64-bundled-distribution# kubectl -n appdynamics-1 get pods NAME READY STATUS RESTARTS AGE appdynamics-operator-7677764d7c-52nx9 1/1 Running 0 51m k8s-cluster-agent-6dbdb6b4dc-dkdgs 1/1 Running 0 30m   Now, what should happen is namespace one-apps should be instrumented with second Cluster Agent aka one running in appdynamics-1 namespace and java-apps should be instrumented with first Cluster Agent aka one running in appdynamics namespace And that is what happened-> root@ip-172-31-12-116:/opt/appdynamics/cluster-agent/cluster-agent-alpine-arm64-bundled-distribution# kubectl -n java-apps describe pod Name: tomcat-app-76df69dc7b-f74wv Namespace: java-apps Priority: 0 Service Account: default Node: ip-172-31-12-116/172.31.12.116 Start Time: Mon, 30 Jun 2025 19:07:11 +0000 Labels: app=tomcat-app pod-template-hash=76df69dc7b Annotations: APPD_DEPLOYMENT_NAME: tomcat-app APPD_INSTRUMENTED_CONTAINERS: tomcat-app APPD_POD_INSTRUMENTATION_STATE: Successful APPD_tomcat-app_APPNAME: tomcat-app-java-apps APPD_tomcat-app_NODEID: 1588804 APPD_tomcat-app_NODENAME: tomcat-app--12 APPD_tomcat-app_TIERID: 30171 APPD_tomcat-app_TIERNAME: tomcat-app cni.projectcalico.org/podIP: 10.244.116.136/32 cni.projectcalico.org/podIPs: 10.244.116.136/32 Status: Running IP: 10.244.116.136 IPs: IP: 10.244.116.136 Controlled By: ReplicaSet/tomcat-app-76df69dc7b Init Containers: appd-agent-attach-java: Container ID: containerd://2785c3422edd754d96270af5e970312791079bb6dc9840acd3ab4af84e5985a0 Image: docker.io/appdynamics/java-agent:latest Image ID: docker.io/appdynamics/java-agent@sha256:d237aeb95a7b77d6e3e5b2c868e03cd22077e24424b58fa2380e6de340305e35 Port: <none> Host Port: <none> Command: /bin/sh -c cp -r /opt/appdynamics/. /opt/appdynamics-java && chown -R 999:999 /opt/appdynamics-java ; ls -la /opt/appdynamics-java State: Terminated Reason: Completed Exit Code: 0 Started: Mon, 30 Jun 2025 19:07:15 +0000 Finished: Mon, 30 Jun 2025 19:07:20 +0000 Ready: True Restart Count: 0 Limits: cpu: 20m memory: 75M Requests: cpu: 10m memory: 50M Environment: <none> Mounts: /opt/appdynamics-java from appd-agent-repo-java (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-95g58 (ro) Containers: tomcat-app: Container ID: containerd://5a27c1bb0d2f539f8288fc65a99cebabeed69d10d34de85c50d0f1389c4e5701 Image: docker.io/abhimanyubajaj98/tomcat-sample Image ID: docker.io/abhimanyubajaj98/tomcat-sample@sha256:19558c877ec1fca506c3b7a6696a7a3a8914e855df3af93778323fb909b2fdff Port: 8080/TCP Host Port: 0/TCP State: Running Started: Mon, 30 Jun 2025 19:07:20 +0000 Ready: True Restart Count: 0 Environment: APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY: <set to the key 'controller-key' in secret 'cluster-agent-secret'> Optional: false JAVA_TOOL_OPTIONS: -Xmx512m -Dappdynamics.agent.accountAccessKey=$(APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY) -Dappdynamics.socket.collection.bci.enable=true -Dappdynamics.jvm.shutdown.mark.node.as.historical=false -Dappdynamics.agent.reuse.nodeName=true -javaagent:/opt/appdynamics-java/javaagent.jar APPDYNAMICS_CONTROLLER_HOST_NAME: controllerces.saas.appdynamics.com APPDYNAMICS_CONTROLLER_PORT: 443 APPDYNAMICS_AGENT_TIER_NAME: tomcat-app APPDYNAMICS_POD_NAMESPACE: java-apps APPDYNAMICS_CONTAINER_NAME: tomcat-app APPDYNAMICS_CONTROLLER_SSL_ENABLED: true APPDYNAMICS_AGENT_ACCOUNT_NAME: controllerces APPDYNAMICS_AGENT_APPLICATION_NAME: tomcat-app-java-apps APPDYNAMICS_JAVA_AGENT_REUSE_NODE_NAME_PREFIX: tomcat-app APPDYNAMICS_CONTAINERINFO_FETCH_SERVICE: cluster-metadata-service.appdynamics:9090 APPDYNAMICS_NETVIZ_AGENT_HOST: (v1:status.hostIP) APPDYNAMICS_NETVIZ_AGENT_PORT: 3892 Mounts: /opt/appdynamics-java from appd-agent-repo-java (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-95g58 (ro) Conditions: Type Status PodReadyToStartContainers True Initialized True Ready True ContainersReady True PodScheduled True Volumes: appd-agent-repo-java: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> kube-api-access-95g58: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: Burstable Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: <none>   Name: tomcat-app-1-6b86c4b444-h65sj Namespace: one-apps Priority: 0 Service Account: default Node: ip-172-31-12-116/172.31.12.116 Start Time: Mon, 30 Jun 2025 19:58:04 +0000 Labels: app=tomcat-app-1 pod-template-hash=6b86c4b444 Annotations: APPD_DEPLOYMENT_NAME: tomcat-app-1 APPD_INSTRUMENTED_CONTAINERS: tomcat-app-1 APPD_POD_INSTRUMENTATION_STATE: Successful APPD_tomcat-app-1_APPNAME: tomcat-app-java-apps-1 APPD_tomcat-app-1_NODEID: 1588843 APPD_tomcat-app-1_NODENAME: tomcat-app-1--1 APPD_tomcat-app-1_TIERID: 31497 APPD_tomcat-app-1_TIERNAME: tomcat-app-1 cni.projectcalico.org/podIP: 10.244.116.150/32 cni.projectcalico.org/podIPs: 10.244.116.150/32 Status: Running IP: 10.244.116.150 IPs: IP: 10.244.116.150 Controlled By: ReplicaSet/tomcat-app-1-6b86c4b444 Init Containers: appd-agent-attach-java: Container ID: containerd://172d5f35af34468b82a4bd22a030495de959a6a71ae79bed17cae8d8d203adff Image: docker.io/appdynamics/java-agent:latest Image ID: docker.io/appdynamics/java-agent@sha256:d237aeb95a7b77d6e3e5b2c868e03cd22077e24424b58fa2380e6de340305e35 Port: <none> Host Port: <none> Command: /bin/sh -c cp -r /opt/appdynamics/. /opt/appdynamics-java && chown -R 999:999 /opt/appdynamics-java ; ls -la /opt/appdynamics-java State: Terminated Reason: Completed Exit Code: 0 Started: Mon, 30 Jun 2025 19:58:05 +0000 Finished: Mon, 30 Jun 2025 19:58:10 +0000 Ready: True Restart Count: 0 Limits: cpu: 20m memory: 75M Requests: cpu: 10m memory: 50M Environment: <none> Mounts: /opt/appdynamics-java from appd-agent-repo-java (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-59w98 (ro) Containers: tomcat-app-1: Container ID: containerd://3234455f370b9af9c2e906ce0a0a183cf9812c53fd2fec3897a4a364c592031c Image: docker.io/abhimanyubajaj98/tomcat-sample Image ID: docker.io/abhimanyubajaj98/tomcat-sample@sha256:19558c877ec1fca506c3b7a6696a7a3a8914e855df3af93778323fb909b2fdff Port: 8080/TCP Host Port: 0/TCP State: Running Started: Mon, 30 Jun 2025 19:58:11 +0000 Ready: True Restart Count: 0 Environment: APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY: <set to the key 'controller-key' in secret 'cluster-agent-secret'> Optional: false JAVA_TOOL_OPTIONS: -Xmx512m -Dappdynamics.agent.accountAccessKey=$(APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY) -Dappdynamics.socket.collection.bci.enable=true -Dappdynamics.jvm.shutdown.mark.node.as.historical=false -Dappdynamics.agent.reuse.nodeName=true -javaagent:/opt/appdynamics-java/javaagent.jar APPDYNAMICS_CONTROLLER_HOST_NAME: ces-controller.saas.appdynamics.com APPDYNAMICS_AGENT_ACCOUNT_NAME: ces-controller APPDYNAMICS_AGENT_APPLICATION_NAME: tomcat-app-java-apps-1 APPDYNAMICS_JAVA_AGENT_REUSE_NODE_NAME_PREFIX: tomcat-app-1 APPDYNAMICS_CONTAINERINFO_FETCH_SERVICE: cluster-metadata-service.appdynamics-1:9090 APPDYNAMICS_CONTROLLER_PORT: 443 APPDYNAMICS_CONTROLLER_SSL_ENABLED: true APPDYNAMICS_AGENT_TIER_NAME: tomcat-app-1 APPDYNAMICS_POD_NAMESPACE: one-apps APPDYNAMICS_CONTAINER_NAME: tomcat-app-1 APPDYNAMICS_NETVIZ_AGENT_HOST: (v1:status.hostIP) APPDYNAMICS_NETVIZ_AGENT_PORT: 3892 Mounts: /opt/appdynamics-java from appd-agent-repo-java (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-59w98 (ro) Conditions: Type Status PodReadyToStartContainers True Initialized True Ready True ContainersReady True PodScheduled True Volumes: appd-agent-repo-java: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> kube-api-access-59w98: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: Burstable Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 32m default-scheduler Successfully assigned one-apps/tomcat-app-1-6b86c4b444-h65sj to ip-172-31-12-116 Normal Pulling 32m kubelet Pulling image "docker.io/appdynamics/java-agent:latest" Normal Pulled 32m kubelet Successfully pulled image "docker.io/appdynamics/java-agent:latest" in 198ms (198ms including waiting). Image size: 84709497 bytes. Normal Created 32m kubelet Created container: appd-agent-attach-java Normal Started 32m kubelet Started container appd-agent-attach-java Normal Pulling 32m kubelet Pulling image "docker.io/abhimanyubajaj98/tomcat-sample" Normal Pulled 32m kubelet Successfully pulled image "docker.io/abhimanyubajaj98/tomcat-sample" in 236ms (236ms including waiting). Image size: 227949061 bytes. Normal Created 32m kubelet Created container: tomcat-app-1 Normal Started 32m kubelet Started container tomcat-app-1   Viola!! This is what we wanted 🙂 ... View more

The AppDynamics Smart Agent offers a powerful way to automate agent installation, updates, and management. While it supports default artifact repositories, you can also host your own agent binaries and use the Custom HTTP URL option to install agents — great for enterprise customization and version control. In this article, we’ll walk through how to host agents in Amazon S3 and use the Smart Agent to install them via a custom HTTP URL.   Prerequisites Before we start, ensure: You have access to an Amazon S3 bucket The Smart Agent is installed on your host The agent ZIP file (e.g., Machine Agent, Java Agent) is available locally   Step 1: Upload the Agent to S3 Use the AWS CLI to upload your agent ZIP file to S3: aws s3 cp machineagent-bundle-64bit-linux-25.4.0.4712.zip s3://your-bucket-name/ Replace your-bucket-name with your actual S3 bucket.   Step 2: Make the Object Publicly Accessible To allow Smart Agent (which runs in AppDynamics cloud) to download the file, it must be public. Allow Public Access via ACL: aws s3api put-object-acl \ --bucket your-bucket-name \ --key machineagent-bundle-64bit-linux-25.4.0.4712.zip \ --acl public-read   Step 3: Get the Public HTTP URL Public S3 objects are accessible at: https://<bucket-name>.s3.<region>.amazonaws.com/<object-key> Example: https://appd-agent-smart.s3.us-west-2.amazonaws.com/machineagent-bundle-64bit-linux-25.4.0.4712.zip   Step 4: Use the URL in Smart Agent Go to AppDynamics Controller → Smart Agent UI Select Install Agent Choose the agent type (e.g., Machine Agent) Under Custom HTTP URL, paste the S3 link       Result Smart Agent will now download the ZIP from your public S3 URL and install the agent on the target host.   ... View more

Overview When monitoring Kubernetes workloads using AppDynamics' Cluster Agent, users often observe a mismatch or confusion between the Memory (MB) values in the table and the Usage% shown in pod detail graphs. This article explains how CPU and memory metrics are computed, displayed, and what to expect when resource limits are or are not defined.   Ask 1: How is CPU% Busy calculated for Pods? AppDynamics uses the following logic: If CPU limits are defined in the pod spec, the CPU%Busy is calculated relative to the defined limit. If limits are not set, AppDynamics falls back to node-level CPU usage, and percentage values may not reflect actual pod utilization.   Ask 2: Does the same logic apply to Memory (MB)? Not exactly. Memory (MB) is always collected as the raw value used by the container. Memory usage % (Usage%) is only calculated and displayed if a memory limit is defined. If no memory limit is set, AppDynamics may still show MB used in the table view, but the Usage% will remain at 0%, as no denominator is available for calculation.   Ask 3: Why does the graph show 0% even when memory is consumed? This behavior is expected when no resource limits are set. AppDynamics cannot calculate a percentage of memory used if there's no defined upper limit to compare against.   Screenshot Summary: Screenshot 1: Pod list view Shows the AppDynamics Operator pod using 11.84 MB memory with 1% CPU.    2. Screenshot 2: Pod container view (with limits defined) Graph shows 11.91% memory usage out of 100Mi defined.          3. Screenshot 3: Pod container view (without limits) NodeJS pod consuming 33 MB memory but graph shows 0% usage.       Verifying Pod Resource Limits: To check if a pod has limits defined: kubectl get pod <pod-name> -n <namespace> -o jsonpath="{.spec.containers[*].resources.limits}"   If output is empty ({}), no limits are set, and usage% will show 0%.   Conclusion: Always refer to the table for actual memory used (MB). Usage% depends on resource limits being set. No bug here – this is by design, and understanding this helps interpret the UI accurately.   Recommendations: Set CPU and memory limits in pod specs for better visibility and scaling insights. Use kubectl top pod to cross-verify values if needed. ... View more

AWS Lambda auto-scales, but every account & region has a concurrency ceiling. When you slam into it CloudWatch emits the Throttles metric. Surfacing that (plus Invocations, Errors, Duration…) in AppDynamics lets you keep all observability in one place and build health rules so you can react before customers feel pain. Pre-requisite AppDynamics Controller Machine Agent EC2 instance profile (IAM role with cloudwatch:ListMetrics and cloudwatch:GetMetricStatistics only) or AWS AccessKey and Secret Java 8 + Maven for building the extension High‑level flow   Install the Machine Agent (recap) wget https://download.appdynamics.com/.../MachineAgent-25.x.x.zip unzip MachineAgent-25.x.x.zip -d /opt/appdynamics # 2 Point to your Controller vim /opt/appdynamics/conf/controller-info.xml # <controller-host>my-controller.company.com</controller-host> # <controller-port>443</controller-port> # <account-name>customer1</account-name> # <account-access-key>abcd123…</account-access-key> # 3 (optional) register as a service sudo systemctl enable appdynamics-machine-agent sudo systemctl start appdynamics-machine-agent Steps are noted here: https://docs.appdynamics.com/appd/24.x/25.4/en/infrastructure-visibility/machine-agent/install-the-machine-agent Build the Custom Namespace extension Extension we are using: https://developer.cisco.com/codeexchange/github/repo/Appdynamics/aws-customnamespace-monitoring-extension/ git clone https://github.com/Appdynamics/aws-customnamespace-monitoring-extension.git cd aws-customnamespace-monitoring-extension mvn clean package cp target/AWSCustomNamespaceMonitor-*.zip \ /opt/appdynamics/monitors/ cd /opt/appdynamics/monitors unzip AWSCustomNamespaceMonitor-*.zip This creates AWSCustomNamespaceMonitor/ with configuration files you need Authoring config.yml metricPrefix: "Custom Metrics|AWS Lambda|" namespace: "AWS/Lambda" accounts: - displayAccountName: "prod" regions: ["us-east-2"] # your Lambda region # keys can stay empty on an EC2 instance profile # awsAccessKey: # awsSecretKey: # optional – pulls every function; give explicit names to narrow it dimensions: - name: "FunctionName" displayName: "Function" values: [".*"] # Display all Lambda functions regionEndPoints: #us-east-1 : monitoring.us-east-1.amazonaws.com us-east-2 : monitoring.us-east-2.amazonaws.com cloudWatchMonitoring: "Basic" concurrencyConfig: noOfAccountThreads: 3 noOfRegionThreadsPerAccount: 3 noOfMetricThreadsPerRegion: 3 threadTimeOut: 30 # Global metrics config for all accounts metricsConfig: # By default, all metrics retrieved from cloudwatch are 'Average' values. # This option allows you to override the metric type. # metricName supports regex # Allowed statTypes are: ave, max, min, sum, samplecount # # Note: Irrespective of the metric type, value will still be reported as # Observed value to the Controller includeMetrics: - name: "Invocations" statType: "sum" delta: false multiplier: 1 aggregationType: "OBSERVATION" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" - name: "Errors" # alias: "Errors" statType: "sum" delta: false multiplier: 1 aggregationType: "OBSERVATION" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" - name: "Duration" # alias: "Duration" statType: "ave" delta: false multiplier: 1 aggregationType: "OBSERVATION" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" - name: "Throttles" # alias: "Throttles" statType: "sum" delta: false multiplier: 1 aggregationType: "OBSERVATION" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" - name: "DeadLetterErrors" # alias: "DeadLetterErrors" statType: "ave" delta: false multiplier: 1 aggregationType: "OBSERVATION" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" - name: "Availability" # alias: "Availability" statType: "ave" delta: false multiplier: 1 aggregationType: "OBSERVATION" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" - name: "IteratorAge" # alias: "IteratorAge" statType: "max" delta: false multiplier: 1 aggregationType: "OBSERVATION" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" metricsTimeRange: startTimeInMinsBeforeNow: 5 endTimeInMinsBeforeNow: 0 # Rate limit ( per second ) for GetMetricStatistics, default value is 400. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html getMetricStatisticsRateLimit: 400 maxErrorRetrySize: 0 All the metrics that you can configure are listed here: Types of metrics for Lambda functions Learn about the different types of metrics for Lambda: invocation metrics, performance metrics, concurrency metrics…docs.aws.amazon.com Restart & verify sudo systemctl restart appdynamics-machine-agent tail -f /opt/appdynamics/logs/machine-agent.log | grep -i "AWS/Lambda" DEBUG AMonitorJob-AWSCustomNamespaceMonitor - Printing Metric [OBSERVATION/AVERAGE/INDIVIDUAL] [Custom Metrics|AWS Lambda|AWS/Lambda|prod|us-east-2|Function|appdshield-deploy-agent|Invocations]=[1]]   With ~10 minutes you can bring Lambda’s key CloudWatch metrics into AppDynamics, build proactive health rules, and keep your SRE team on a single pane of glass. Extend the same approach to other AWS namespaces or use AppDynamics’ Lambda‑specific extension if you want the full metric set out‑of‑the‑box. Happy monitoring! ... View more

How to Deploy AppDynamics Database Agent for MySQL on OpenShift Introduction: Monitoring database performance is crucial for ensuring application stability and efficiency. This guide walks you through deploying the AppDynamics Database Agent on OpenShift to monitor a MySQL database. You'll learn how to: Set up a MySQL instance on OpenShift Deploy the AppDynamics Database Agent with proper permissions Configure MySQL monitoring in the AppDynamics Controller UI Verify metrics and troubleshoot potential issues By the end of this guide, you’ll have full visibility into your MySQL performance, including query analysis, resource utilization, and overall health monitoring in AppDynamics.   1. Prerequisites OpenShift Cluster: Access via the oc CLI or the OpenShift Web Console. AppDynamics Controller: You have the host, port, account name, access key. AppDynamics License: Ensure Database Monitoring (MySQL) is licensed. Database Agent Image: Typically docker.io/appdynamics/db-agent or a registry provided by AppDynamics. Basic MySQL Docker image: For this example, we’ll use mysql:8.0 from Docker Hub. 2. Create/Open a New Project In the CLI, for example: oc new-project mydb-project oc project mydb-project 3. Deploy a MySQL Instance on OpenShift Below is a simple YAML for a Deployment and Service. In a real environment, you’d likely set up persistent storage, handle secrets for credentials, etc. # mysql.yaml apiVersion: apps/v1 kind: Deployment metadata: name: mysql-deployment spec: replicas: 1 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: mysql image: mysql:8.0 env: - name: MYSQL_ROOT_PASSWORD value: rootpassword - name: MYSQL_DATABASE value: sampledb - name: MYSQL_USER value: appduser - name: MYSQL_PASSWORD value: appdpass ports: - containerPort: 3306 --- apiVersion: v1 kind: Service metadata: name: mysql-service spec: selector: app: mysql ports: - protocol: TCP port: 3306 targetPort: 3306 Apply it: oc apply -f mysql.yaml Check that the pod and service are running: oc get pods oc get svc You should see mysql-deployment-... running and a mysql-service on port 3306 4. Create a Secret for AppDynamics Credentials Store your AppDynamics account access key securely. For example: oc create secret generic appd-secret \ --from-literal=APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY=<YOUR-ACCESS-KEY> This secret will be referenced by the Database Agent deployment. 5. Grant the “anyuid” SCC The AppDynamics Database Agent container often tries to run a startup script (start-appdynamics-database-monitoring). In OpenShift’s default restricted SCC, containers run as a random non-root UID. If the script in the image doesn’t have proper permissions for non-root, you get a permission denied error. 5.1 Create a Service Account (Recommended) oc create serviceaccount appd-db-agent-sa 5.2 Give anyuid SCC to That Service Account oc adm policy add-scc-to-user anyuid -z appd-db-agent-sa -n mydb-project Now, the service account appd-db-agent-sa is allowed to run images that require root or any specific UID. Note: This is a quick fix and less secure. A better approach is to build a custom image or ensure the official image has proper file permissions (so it can run under restricted SCC). But for simplicity, we’ll use anyuid in this guide. 6. Deploy the AppDynamics Database Agent Here’s a minimal YAML that uses environment variables (rather than a config file) to configure the agent. It references the secret for the access key and uses the appd-db-agent-sa service account we just created. # appd-db-agent.yaml apiVersion: apps/v1 kind: Deployment metadata: name: appd-database-agent spec: replicas: 1 selector: matchLabels: app: appd-database-agent template: metadata: labels: app: appd-database-agent spec: serviceAccountName: appd-db-agent-sa # <-- Important! containers: - name: db-agent image: docker.io/appdynamics/db-agent:latest env: - name: APPDYNAMICS_CONTROLLER_HOST_NAME value: "my-controller.example.com" - name: APPDYNAMICS_CONTROLLER_PORT value: "443" - name: APPDYNAMICS_CONTROLLER_SSL_ENABLED value: "true" - name: APPDYNAMICS_AGENT_ACCOUNT_NAME value: "myAccount" - name: APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY valueFrom: secretKeyRef: name: appd-secret key: APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY # Name this agent as it appears in the Controller - name: APPDYNAMICS_DB_AGENT_NAME value: "MyOpenShiftDBAgent" - name: APPDYNAMICS_DB_AGENT_TIER_NAME value: "MyDBTier" - name: APPDYNAMICS_DB_AGENT_NODE_NAME value: "MyDBNode" # For memory management or other Java opts - name: JAVA_OPTS value: "-Xms128m -Xmx512m" Apply it: oc apply -f appd-db-agent.yaml Check pod logs: oc get pods oc logs deployment/appd-database-agent If all goes well, you should see logs indicating it connected to the Controller. If you get a permission denied error, confirm you’re using the serviceAccountName: appd-db-agent-sa and that the SA has the anyuid SCC. 7. Configure MySQL Monitoring in the Controller UI By default, if you only used environment variables, the agent knows how to contact the Controller but doesn’t yet know which MySQL host to monitor. We create a MySQL collector in the AppDynamics UI: Open the AppDynamics Controller in your browser. Navigate to Databases → Agents. Find your agent (e.g., MyOpenShiftDBAgent) and click on it. In the Databases tab, click Add → Add MySQL Collector. Collector Configuration: Display Name: e.g. MyOpenShiftDBAgent Host: mysql-service (assuming your Database Agent is in the same namespace as mysql-service), If in a different namespace, you might need the FQDN: mysql-service.mydb-project.svc.cluster.local Port: 3306 Username: appduser Password: appdpass (Optionally) Tweak advanced fields like Collection Interval, Lock Level, Query Analysis, etc.   Save the collector. Within a couple of minutes, the Database Agent should start collecting MySQL metrics. 8. Verify and View Metrics Once the MySQL collector is set up: In the AppDynamics Controller, go to Databases → Databases. You’ll see a row for your new MySQL collector (e.g., MyOpenShiftDBAgent). Click it to see: Overview: CPU, memory, throughput. Queries: Slow queries, wait stats, top queries, etc. Resources: For deeper performance insights.   Conclusion Deploying the AppDynamics Database Agent alongside MySQL on OpenShift involves: Creating a project, a MySQL deployment, and a service. Setting up the Database Agent container, ensuring it can run the startup script (via SCC or a custom image). Creating a MySQL collector in AppDynamics to specify connection details. Once configured, you’ll see full visibility into your MySQL instance — top queries, performance metrics, and more — in the AppDynamics Controller. Feel free to add screenshots to illustrate each step in both the OpenShift Web Console and the AppDynamics UI. ... View more

Guide to Monitoring URLs with Authentication Using Splunk AppDynamics and Python Monitoring URLs are an important part of your FullStackMonitoring. Splunk AppDynamics lets you monitor URLs with different authentication. In this article, we will create a simple URL with a username and password. Afterwards, we will monitor it using AppDynamics Machine Agent. Create a Simple API with Python (Flask) Install Flask: pip install flask​ Create the API: Save the following Python code to a file, e.g., basic_auth_api.py : from flask import Flask, request, jsonify from flask_httpauth import HTTPBasicAuth app = Flask(__name__) auth = HTTPBasicAuth() # Dummy users for authentication users = { "user1": "password123", "user2": "securepassword", } @auth.get_password def get_pw(username): return users.get(username) @app.route('/api/data', methods=['GET']) @auth.login_required def get_data(): return jsonify({"message": f"Hello, {auth.username()}! Here is your data."}) if __name__ == '__main__': app.run(debug=True, port=5000) Run the API: Start the server by running: python basic_auth_api.py Test the API: Use curl to access the API: curl -u user1:password123 http://127.0.0.1:5000/api/data You should see a response like this: { "message": "Hello, user1! Here is your data." }​ Install Machine Agent You can install the Machine agent as recommended here Setup URL Monitoring Extension Clone the Github Repo: git clone https://github.com/Appdynamics/url-monitoring-extension.git​ cd url-monitoring-extension​ Download and install Apache Maven which is configured with  Java 8  to build the extension artifact from the source. You can check the Java version used in Maven using command  mvn -v  or  mvn --version . If your maven is using some other Java version then please download Java 8 for your platform and set JAVA_HOME parameter before starting maven. Run below in url-monitoring-extension directory mvn clean install Go into the target directory and copy the UrlMonitor-2.2.1.zip, Afterwards unzip the content inside <MA-Home>/monitors/folder cd target/ mv UrlMonitor-2.2.1.zip /opt/appdynamics/machine-agent/monitors unzip UrlMonitor-2.2.1.zip​ This will create an UrlMonitor directory inside the Monitors folder. Monitor the URL Inside the UrlMonitor folder, edit the config.yml file Under sites, I have added: sites: - name: AppDynamics url: http://127.0.0.1:5000/api/data username: user1 password: password123 authType: BASIC​ Change: metricPrefix: "Custom Metrics|URL Monitor|"​ Now, All you need to do is Start your Machine Agent again. Afterward, you can see this URL monitor in your AppDynamics Controller. ... View more

A Step-by-Step Guide to Setting Up and Monitoring Redis with AppDynamics on Ubuntu EC2 Monitoring your Redis instance is essential for ensuring optimal performance and identifying potential bottlenecks in real-time. In this guide, we’ll walk through the process of setting up Redis on an Ubuntu EC2 instance and configuring SplunkAppDynamics Redis Monitoring Extension to capture key metrics. Step 1: Setting up Redis on Ubuntu Prerequisites An AWS account with an EC2 instance running Ubuntu. SSH access to your EC2 instance. Installing Redis Update package lists and install Redis: sudo apt-get update sudo apt-get install redis-server​ Verify the installation: redis-server --version​ Ensure Redis is running: sudo systemctl status redis​ Step 2: Installing AppDynamics Machine Agent Download the Machine Agent: Visit AppDynamics and download the Machine Agent for your environment. Install the Machine Agent: Follow the installation steps provided in the AppDynamics Machine Agent documentation. https://docs.appdynamics.com/appd/24.x/24.11/en/infrastructure-visibility/machine-agent/install-the-machine-agent Verify Installation: Start the Machine Agent and confirm it connects to your AppDynamics Controller. Step 3: Configuring AppDynamics Redis Monitoring Extension Clone the Redis Monitoring Extension Repository git clone https://github.com/Appdynamics/redis-monitoring-extension.git cd redis-monitoring-extension Build the Extension sudo apt-get install openjdk-8-jdk maven mvn clean install Locate the  .zip file in the target folder and extract it: unzip target/RedisMonitor-*.zip -d <MachineAgent_Dir>/monitors/ Edit the Configuration File Navigate to the extracted folder and edit config.yml : metricPrefix: "Custom Metrics|Redis" #Add your list of Redis servers here. servers: - name: "localhost" host: "localhost" port: "6379" password: "" #encryptedPassword: "" useSSL: false Restart the Machine Agent .<MachineAgent_Dir>/bin/machine-agent Step 4: Verifying Metrics in AppDynamics Log in to your AppDynamics Controller. Navigate to the Metric Browser. Look for metrics under the path: Custom Metrics|Redis Verify that metrics like used_memory , connected_clients , and keyspace_hits are visible. Conclusion By combining the power of Redis with the advanced monitoring capabilities of AppDynamics, you can ensure your application remains scalable and responsive under varying workloads. Whether you’re troubleshooting an issue or optimizing performance, this setup gives you full visibility into your Redis instance. If you found this guide helpful, please share and connect with me for more DevOps insights! ... View more

Comprehensive Guide to RabbitMQ Setup, Integration with Python, and Monitoring with AppDynamics Introduction RabbitMQ is a powerful open-source message broker that supports a variety of messaging protocols, including AMQP. It allows developers to build robust, scalable, and asynchronous messaging systems. However, to ensure optimal performance, monitoring RabbitMQ metrics is crucial. This tutorial walks you through setting up RabbitMQ, integrating it with a Python application, and monitoring its metrics using AppDynamics. Step 1: Setting Up RabbitMQ 1.1 Install RabbitMQ via Docker To quickly get RabbitMQ up and running, use the official RabbitMQ Docker image with the management plugin enabled. Run the following command to start RabbitMQ: docker run -d --hostname my-rabbit --name rabbitmq \ -e RABBITMQ_DEFAULT_USER=guest \ -e RABBITMQ_DEFAULT_PASS=guest \ -p 5672:5672 -p 15672:15672 \ rabbitmq:management Management Console: Accessible at http://localhost:15672 . Default Credentials: Username: guest Password: guest 1.2 Verify the Setup Once the container is running, verify the RabbitMQ server by accessing the Management Console in your browser. Alternatively, test the API endpoint: curl -u guest:guest http://localhost:15672/api/overview This should return RabbitMQ metrics in JSON format. Step 2: Writing a Simple RabbitMQ Producer and Consumer in Python 2.1 Install Required Library Install the pika library for Python, which is used to interact with RabbitMQ: pip install pika 2.2 Create the Producer Script ( send.py ) This script connects to RabbitMQ, declares a queue, and sends a message. import pika # Connect to RabbitMQ connection = pika.BlockingConnection(pika.ConnectionParameters('localhost')) channel = connection.channel() # Declare a queue channel.queue_declare(queue='hello') # Publish a message channel.basic_publish(exchange='', routing_key='hello', body='Hello RabbitMQ!') print(" [x] Sent 'Hello RabbitMQ!'") connection.close() 2.3 Create the Consumer Script ( receive.py ) This script connects to RabbitMQ, consumes messages from the queue, and prints them. import pika # Connect to RabbitMQ connection = pika.BlockingConnection(pika.ConnectionParameters('localhost')) channel = connection.channel() # Declare a queue channel.queue_declare(queue='hello') # Define a callback to process messages def callback(ch, method, properties, body): print(f" [x] Received {body}") channel.basic_consume(queue='hello', on_message_callback=callback, auto_ack=True) print(' [*] Waiting for messages. To exit press CTRL+C') channel.start_consuming() 2.4 Test the Application a. Run the consumer in one terminal: python3 receive.py b. Send a message from another terminal: python3 send.py c. Observe the message output in the consumer terminal. [x] Sent 'Hello RabbitMQ!' [x] Received b'Hello RabbitMQ!' Step 3: Monitoring RabbitMQ with AppDynamics 3.1 Configure RabbitMQ Management Plugin Ensure that the RabbitMQ Management Plugin is enabled (default in the Docker image). It exposes an HTTP API that provides metrics. 3.2 Create a Custom Monitoring Script Use a shell script to fetch RabbitMQ metrics and send them to the AppDynamics Machine Agent. script.sh #!/bin/bash # RabbitMQ Management API credentials USERNAME="guest" PASSWORD="guest" URL="http://localhost:15672/api/overview" # Fetch metrics from RabbitMQ Management API RESPONSE=$(curl -s -u $USERNAME:$PASSWORD $URL) if [[ $? -ne 0 || -z "$RESPONSE" ]]; then echo "Error: Unable to fetch RabbitMQ metrics" exit 1 fi MESSAGES=$(echo "$RESPONSE" | jq '.queue_totals.messages // 0') MESSAGES_READY=$(echo "$RESPONSE" | jq '.queue_totals.messages_ready // 0') DELIVER_GET=$(echo "$RESPONSE" | jq '.message_stats.deliver_get // 0') echo "name=Custom Metrics|RabbitMQ|Total Messages, value=$MESSAGES" echo "name=Custom Metrics|RabbitMQ|Messages Ready, value=$MESSAGES_READY" echo "name=Custom Metrics|RabbitMQ|Deliver Get, value=$DELIVER_GET" 3.3 Integrate with AppDynamics Machine Agent Place the Script: Copy the script.sh script to the Machine Agent monitors directory: cp script.sh <MachineAgent_Dir>/monitors/RabbitMQMonitor/ 2. Create monitor.xml : Create a monitor.xml file to configure the Machine Agent: <monitor> <name>RabbitMQ</name> <type>managed</type> <enabled>true</enabled> <enable-override os-type="linux">true</enable-override> <description>RabbitMQ </description> <monitor-configuration> </monitor-configuration> <monitor-run-task> <execution-style>periodic</execution-style> <name>Run</name> <type>executable</type> <task-arguments> </task-arguments> <executable-task> <type>file</type> <file>script.sh</file> </executable-task> </monitor-run-task> </monitor> 3. Restart the Machine Agent: Restart the agent to apply the changes: cd <MachineAgent_Dir>/bin ./machine-agent & Step 4: Viewing Metrics in AppDynamics Log in to your AppDynamics Controller. Navigate to Servers > Custom Metrics. Look for metrics under: Custom Metrics|RabbitMQ You should see metrics like: Total Messages Messages Ready Deliver Get ... View more

How to Redirect Smart Agent Temporary Files to Avoid /tmp Space Limitations on Linux Installing any APM or Machine agent with Smart Agent on your Linux box will use /tmp directory to copy agent binaries before moving them to the intended directory. The problem You can get an error like below: error message = Error extracting Machine Agent in staging: reading file in zip archive: /tmp/.staging/machine-agent/jre/lib/modules: writing file: write /tmp/.staging/machine-agent/jre/lib/modules: no space left on device Error creating Machine Agent service: error installing service: error moving service file to destination: rename /tmp/.staging/appdynamics-machine-agent.service /etc/systemd/system/appdynamics-machine-agent.service: invalid cross-device link These errors are caused due to less space in /tmp folder or /tmp directory mounted on an external device. How to fix it You need to have Smart Agent use any other directory on our host then tmp. To do this: Go to <Smart-Agent-Home-Directory>, In my case the directory is /opt/appdynamics/appdsmartagent Commands in order: cd /opt/appdynamics/appdsmartagent ./smartagentctl stop export TMPDIR=/opt/appdynamics ./smartagentctl start Now in your logs, you will see {"severityText":"INFO","timestamp":"2024-10-04T16:30:29.692Z","name":"native","caller":"machine/task_helper.go:48","body":"downloaded file to ","downloaded file":"/opt/appdynamics/.staging/download/machineagent-bundle-64bit-linux-24.9.0.4408.zip"} {"severityText":"INFO","timestamp":"2024-10-04T16:30:29.692Z","name":"native","caller":"machine/task_helper.go:161","body":"Extracting zip","package.name":"8a5e85401b3a01ac5dadd6394c235dbf032ffa04;MACHINE_AGENT","src path":"/opt/appdynamics/.staging/download/machineagent-bundle-64bit-linux-24.9.0.4408.zip","dest path":"/opt/appdynamics/.staging/machine-agent"} This means, Smart Agent is now copying everything in /opt/appdynamics directory. ... View more

Currently, InfraViz doesn't let you deploy Custom extensions. If you wish to deploy custom extensions on Kubernetes using machine agents then this article is for you. This can be done in 2 ways: Creating a new Machine Agent Image Creating a new yaml file for Machine Agent Creating a new Machine Agent Image Now if you wish to use this method, which is modifying the Machine Agent image you need to take a step back and ask yourself: Do you need the extension on all nodes? If not, then if you deploy InfraViz in default by just updating the Image with extension then on the node where it works, everything will be fine but on others, you will have logs filled with ERROR/WARN messages which can potentially lead to Machine agent collector script timing out. Do you need a Machine Agent on all nodes? If not, then we are okay. We can use NodeSelector property of InfraViz and simply deploy this new Image using InfraViz on the specific node. In any case, the Dockerfile will look like below: FROM ubuntu:latest # Install curl and unzip RUN apt-get update && apt-get install -y curl unzip procps # Add and unzip the Machine Agent bundle ADD machineagent-bundle-64bit-linux-23.7.0.3689.zip /tmp/machineagent.zip RUN unzip /tmp/machineagent.zip -d /opt/appdynamics && rm /tmp/machineagent.zip # Set environment variable for Machine Agent home ENV MACHINE_AGENT_HOME /opt/appdynamics # Add AWS API Gateway Monitor and start-appdynamics script ADD create-open-file-extension-folder /opt/appdynamics/monitors ADD start-appdynamics ${MACHINE_AGENT_HOME} # Make start-appdynamics script executable RUN chmod 744 ${MACHINE_AGENT_HOME}/start-appdynamics # Set Java Home environment variable ENV JAVA_HOME /opt/appdynamics/jre/bin/java # Run AppDynamics Machine Agent CMD ["/opt/appdynamics/start-appdynamics"] NOTE: In the same directory as Dockerfile You need to have the appdynamics zip in your local. In my case, I have machineagent-bundle-64bit-linux-23.7.0.3689.zip in my local create-open-file-extension-folder is the extension folder which I am moving to /opt/appdynamics/monitors, This has my script.sh and monitor.xml file Remember for extensions, the Machine agent looks for folders and files in monitors directory. start-appdynamics.sh script. This is the content of start-appdynamics.sh script. You will need to edit it and add your Controller configuration. MA_PROPERTIES="-Dappdynamics.controller.hostName=xxx.saas.appdynamics.com" MA_PROPERTIES+=" -Dappdynamics.controller.port=443" MA_PROPERTIES+=" -Dappdynamics.agent.accountName=xxxx" MA_PROPERTIES+=" -Dappdynamics.agent.accountAccessKey=xx" MA_PROPERTIES+=" -Dappdynamics.controller.ssl.enabled=true" MA_PROPERTIES+=" -Dappdynamics.sim.enabled=true" MA_PROPERTIES+=" -Dappdynamics.docker.enabled=false" MA_PROPERTIES+=" -Dappdynamics.docker.container.containerIdAsHostId.enabled=true" # Start Machine Agent ${MACHINE_AGENT_HOME}/jre/bin/java ${MA_PROPERTIES} -jar ${MACHINE_AGENT_HOME}/machineagent.jar Great. Now all you need to do is build the Image and push it to your repository. Once done, in the InfraViz section, Update the Image section of InfraViz with this new Image Creating a new yaml file for Machine Agent Now the second option is using deployment and InfraViz together. I have created a infraviz-deployment.yaml file. This is a deployment that I am deploying on a specific node. apiVersion: apps/v1 kind: Deployment metadata: name: machine-agent-extension labels: app: machine-agent-extension spec: replicas: 1 selector: matchLabels: app: machine-agent-extension template: metadata: labels: app: machine-agent-extension spec: initContainers: - name: create-open-file-extension-folder image: busybox command: ['sh', '-c', 'mkdir -p /opt/appdynamics/monitors/open-file-extension && cp /tmp/config/* /opt/appdynamics/monitors/open-file-extension && chmod +x /opt/appdynamics/monitors/open-file-extension/script.sh'] volumeMounts: - name: config-volume mountPath: /tmp/config # Mount ConfigMap here temporarily - name: open-file-extension mountPath: /opt/appdynamics/monitors/open-file-extension # Target directory in emptyDir containers: - name: machine-agent-extension image: appdynamics/machine-agent:latest ports: - containerPort: 9090 env: - name: APPDYNAMICS_CONTROLLER_HOST_NAME value: "xxxx.saas.appdynamics.com" - name: APPDYNAMICS_CONTROLLER_PORT value: "443" - name: APPDYNAMICS_AGENT_ACCOUNT_NAME value: "xxx" - name: APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY value: "xxx" - name: APPDYNAMICS_SIM_ENABLED value: "true" - name: APPDYNAMICS_CONTROLLER_SSL_ENABLED value: "true" volumeMounts: - name: open-file-extension mountPath: /opt/appdynamics/monitors/open-file-extension volumes: - name: config-volume configMap: name: open-file-extension-config # ConfigMap holding script.sh and monitor.xml - name: open-file-extension emptyDir: {} # EmptyDir to allow read/write nodeSelector: kubernetes.io/hostname: "ip-222-222-222-222.us-west-2.compute.internal" --- apiVersion: v1 kind: ConfigMap metadata: name: open-file-extension-config namespace: default data: script.sh: | #!/bin/bash # Get the current open files limit for the process open_files_limit=$(ulimit -n) ##Commentlineforcheck # Output the open files limit to stdout echo "name=Custom Metrics|OpenFilesLimitMonitor|OpenFilesLimit,value=$open_files_limit" monitor.xml: | <monitor> <name>OpenFile</name> <type>managed</type> <enabled>true</enabled> <enable-override os-type="linux">true</enable-override> <description>OpenFile</description> <monitor-configuration></monitor-configuration> <monitor-run-task> <execution-style>periodic</execution-style> <name>Run</name> <type>executable</type> <task-arguments></task-arguments> <executable-task> <type>file</type> <file>script.sh</file> </executable-task> </monitor-run-task> </monitor> Right now I am only monitoring one node though, so we will use InfraViz to monitor other nodes now. Taint the nodes that I don't want InfraViz to run now which is going to be above one ^ kubectl taint node ip-222-222-222-222.us-west-2.compute.internal machine-agent=false:NoSchedule Now, I can deploy InfraViz.yaml normally and it won't be deployed on ip-222 node. Now you have MA running on all of your nodes, one with extension, and the rest normally Please reach out to Support if you have any questions. ... View more

Step-by-Step Guide to Deploying AppDynamics Smart Agent Using Ansible on Linux Systems This article will guide you through installing the AppDynamics Smart Agent on a Linux system using Ansible. It covers downloading, configuring, and starting the Smart Agent in an automated fashion. This setup ensures that the Smart Agent is correctly configured for your environment. Prerequisites: Ansible Installed: Make sure Ansible is installed on the machine where you are running the playbook. Sudo Privileges: The playbook requires sudo (root) privileges to execute tasks. Download URL: You need a valid download URL for the AppDynamics Smart Agent. You can get this from the AppDynamics download site (replace the provided URL with your own). Steps to Set Up the Ansible Playbook 1. Directory Structure Create a directory structure for the Ansible playbook as follows: 2. Inventory File Define your target machine (localhost in this case) in the inventory file: [appd_agents] localhost ansible_connection=local 3. Ansible Playbook ( playbook.yml ) The main playbook references the smart_agent role. Ensure become: true is set to allow privilege escalation for necessary tasks: --- - hosts: appd_agents become: true roles: - smart_agent 4. Role Tasks ( roles/smart_agent/tasks/main.yml ) The tasks in this role will include downloading, unarchiving, configuring, and starting the Smart Agent. - name: Download AppDynamics Smart Agent using curl command: > curl -L -O -H "Authorization: Bearer <YOUR_AUTH_TOKEN>" "https://download.appdynamics.com/download/prox/download-file/appdsmartagent/<version>/appdsmartagent_64_linux_<version>.zip" args: chdir: /tmp - name: Unarchive the Smart Agent zip unarchive: src: /tmp/appdsmartagent_64_linux_<version>.zip dest: /opt/appdynamics/ remote_src: yes - name: Configure Smart Agent in config.ini replace: path: /opt/appdynamics/config.ini regexp: 'ControllerURL\s*=\s*.*' replace: 'ControllerURL=https://xxxxx.saas.appdynamics.com' - name: Set ControllerPort in config.ini replace: path: /opt/appdynamics/config.ini regexp: 'ControllerPort\s*=\s*.*' replace: 'ControllerPort=443' - name: Set FMServicePort in config.ini replace: path: /opt/appdynamics/config.ini regexp: 'FMServicePort\s*=\s*.*' replace: 'FMServicePort=443' - name: Set AccountAccessKey in config.ini replace: path: /opt/appdynamics/config.ini regexp: '^AccountAccessKey\s*=\s*.*' replace: 'AccountAccessKey=<YOUR_ACCOUNT_ACCESS_KEY>' - name: Ensure AccountName is set in the main section of config.ini lineinfile: path: /opt/appdynamics/config.ini regexp: '^AccountName\s*=' line: 'AccountName=xxxxx' insertafter: '^ControllerPort\s*=.*' - name: Enable SSL in config.ini replace: path: /opt/appdynamics/config.ini regexp: 'EnableSSL\s*=\s*.*' replace: 'EnableSSL=true' - name: Start Smart Agent shell: /opt/appdynamics/smartagentctl start --service > /tmp/log.log 2>&1 become: yes register: output 5. Replace the Download URL and other controller parameter Replace the download URL placeholder with your own Smart Agent download URL from the AppDynamics download site. In the command task for downloading the Smart Agent, replace <YOUR_AUTH_TOKEN> with your AppDynamics authentication token and replace <version> with the appropriate version for your Smart Agent. For example: - name: Download AppDynamics Smart Agent using curl command: > curl -L -O -H "Authorization: Bearer YOUR_AUTH_TOKEN" "https://download.appdynamics.com/download/prox/download-file/appdsmartagent/24.8.0.551/appdsmartagent_64_linux_24.8.0.551.zip" args: chdir: /tmp Replace Controller URL, Controller Port, AccessKey and AccountName with your credentials 6. Running the Playbook To run the playbook, execute the following command: ansible-playbook -i inventory playbook.yml Conclusion This Ansible playbook simplifies downloading, configuring, and running the AppDynamics Smart Agent on a Linux system. Make sure to replace the download URL and account details with your specific values, and you’ll have the agent up and running in no time. ... View more

Step-by-Step Guide to Setting Up AppDynamics Smart Agent on Windows Systems Go to the AppDynamics Downloads area in Accounts On the Agent tab, under the area "Type"  Select "Agent Management" and then download the AppDynamics Smart Agent for Windows For Windows, you require Administrator access to start Smart Agent. Therefore, you cannot start Smart Agent as a process in Windows Once you have downloaded the Smart Agent on Windows box, unzip the content The unzipped content will have the below files as of version 24.8.0 of AppDynamics Smart Agent You need to edit the config.ini file, specifically the below section: ControllerURL=<Your-AppDynamics-Controller-Url> ControllerPort=<Your-AppDynamics-Controller-port> FMServicePort=<Your-AppDynamics-Controller-port> AgentType = <Let-this-be-null> AccountAccessKey=<Your-AppDynamics-Controller-accessKey> AccountName=<Your-AppDynamics-Controller-Account-name> EnableSSL= <True-If-SSL-Is-Enabled-Else-False> Once this is edited, open the CMD prompt go the directory where SmartAgent is downloaded, and run: smartagentctl start --service​ Once this is done, SmartAgent should be installed as a service named “appdsmartagent”. You can confirm this from TaskManager.  Now, if you go to the AppDynamics Controller UI, under Agent Management -> Smart Agent, you will be able to see Smart Agent installed. ... View more

AppDynamics Cluster Agent allows you to auto-instrument your Applications running on Kubernetes. The auto-instrumentation injects APM agents on runtime, modifying your deployment spec with an init-container of AppDynamics APM agent. You can use different strategies to target Kubernetes deployments, StatefulSet, or DeploymentConfigs. In this article, we will cover instrumenting one deployment by using a label defined on the Deployment level (it will be the same for DeploymentConfig and StatefulSet) Let's take this forward with two sample deployments running in namespace abhi-java-apps-second. My first deployment: --- apiVersion: apps/v1 kind: Deployment metadata: name: tomcat-app-abhi labels: app: tomcat-app-abhi-second namespace: abhi-java-apps-second spec: replicas: 1 selector: matchLabels: app: tomcat-app-abhi template: metadata: labels: app: tomcat-app-abhi spec: containers: - name: tomcat-app-abhi #image: docker.io/abhimanyubajaj98/java-tomcat-sample-app-buildx:latest image: docker.io/abhimanyubajaj98/java-application:latest imagePullPolicy: Always ports: - containerPort: 8080 env: - name: JAVA_TOOL_OPTIONS value: -Xmx512m #- name: APPDYNAMICS_AGENT_UNIQUE_HOST_ID # value: $(cat /proc/self/cgroup | head -1 | awk -F '/' '{print $NF}' | cut -c 16-27) --- apiVersion: v1 kind: Service metadata: name: tomcat-app-service labels: app: tomcat-app-abhi namespace: abhi-java-apps-second spec: ports: - port: 8080 targetPort: 8080 selector: app: tomcat-app-abhi My Second deployment: --- apiVersion: apps/v1 kind: Deployment metadata: name: tomcat-app-abhi-labelmatch labels: app: tomcat-app-abhi-second-labelmatch namespace: abhi-java-apps-second spec: replicas: 1 selector: matchLabels: app: tomcat-app-abhi-labelmatch template: metadata: labels: app: tomcat-app-abhi-labelmatch spec: containers: - name: tomcat-app-abhi #image: docker.io/abhimanyubajaj98/java-tomcat-sample-app-buildx:latest image: docker.io/abhimanyubajaj98/java-application:latest imagePullPolicy: Always ports: - containerPort: 8080 env: - name: JAVA_TOOL_OPTIONS value: -Xmx512m #- name: APPDYNAMICS_AGENT_UNIQUE_HOST_ID # value: $(cat /proc/self/cgroup | head -1 | awk -F '/' '{print $NF}' | cut -c 16-27) --- apiVersion: v1 kind: Service metadata: name: tomcat-app-service-labelmatch labels: app: tomcat-app-abhi-labelmatch namespace: abhi-java-apps-second spec: ports: - port: 8080 targetPort: 8080 selector: app: tomcat-app-abhi-labelmatch Now, my use case is only for instrument deployment tomcat-app-abhi-labelmatch. To do this, I would need to edit my cluster-agent.yaml and add the below specs: instrumentationRules: - namespaceRegex: abhi-java-apps-second labelMatch: - app: tomcat-app-abhi-second-labelmatch tierName: abhiapps language: java imageInfo: image: "docker.io/appdynamics/java-agent:latest" agentMountPath: /opt/appdynamics imagePullPolicy: Always Now, after the deployment is done, only the deployment tomcat-app-abhi-labelmatch will have AppDynamics Java Agent. ... View more

Introduction In Kubernetes, securing communications between different components is essential for maintaining the integrity and confidentiality of your applications. Certificates play a pivotal role in this security ecosystem, enabling encrypted communication, and authentication, and ensuring that data remains tamper-proof. However, as Kubernetes environments grow in complexity, tracking which certificates are used by specific deployments can become challenging. This guide provides a comprehensive, secrets-focused approach to identifying and verifying the certificates used by a specific Kubernetes deployment, using the AppDynamics Cluster Agent as a practical example. Understanding Kubernetes Certificates What Are Certificates in Kubernetes? Certificates in Kubernetes are digital documents used to secure communications between different components, such as API servers, nodes, and deployed applications. These certificates are typically X.509 certificates and are used for encryption, authentication, and integrity checks. In Kubernetes, certificates can be issued by a Certificate Authority (CA) and are often managed automatically, although there are scenarios where manual intervention is required. What Are Secrets in Kubernetes? Kubernetes Secrets are objects used to store sensitive information, such as passwords, OAuth tokens, and, importantly, TLS certificates. Storing certificates as secrets allows Kubernetes to manage and distribute them securely to the pods that require them. Common Certificate Use Cases Some common uses of certificates in Kubernetes include: Securing API Server Communications: Ensuring that communications between the Kubernetes API server and nodes are encrypted. Ingress Controllers: Handling SSL/TLS termination for external traffic entering the cluster. Mutual TLS (mTLS) Between Services: Encrypting communication between microservices within the cluster. Application Routes: Securing external access to applications through routes or services. Understanding Kubernetes Secrets Types of Secrets: kubernetes.io/tls: Used specifically for storing TLS certificates. Opaque: A generic type of secret that can store any key-value pair, including certificates. kubernetes.io/dockercfg: Used for storing Docker configuration, particularly for private image registries. kubernetes.io/service-account-token: Used to manage service account tokens. Prerequisites Before you begin, ensure you have the following: kubectl: The command-line tool for interacting with Kubernetes clusters. OpenSSL: A tool for working with SSL/TLS certificates. Access: Sufficient permissions to access and manage secrets in the Kubernetes cluster. AppDynamics Cluster Agent installed: Install the Cluster Agent  Step-by-Step Guide to Identifying Certificates via Secrets Step 1: List All Secrets in the Namespace Certificates are stored within secrets. Start by listing all secrets in the relevant namespace to identify potential candidates. kubectl get secrets -n <namespace> In our case I have deployed Cluster agent in appdynamics namespace, So I will reference appdynamics in subsequent steps. Step 2: Filter Secrets Related to Certificates Kubernetes secrets that store TLS certificates usually have the type kubernetes.io/tls . Filter secrets by this type to narrow down your search. kubectl get secrets -n appdynamics -o json | jq '.items[] | select(.type=="kubernetes.io/tls") | .metadata.name' In AppDynamics, We don’t have a TLS secret by default so the above should return nothing. Step 3: Check Other Secrets If no kubernetes.io/tls secrets are found, inspect other secrets in the namespace to see if they contain certificates. This command will list all the secrets in the namespace, including those that might contain certificate data. kubectl get secrets -n appdynamics NAME TYPE DATA AGE appdynamics-cluster-agent-dockercfg-zvglk kubernetes.io/dockercfg 1 30m appdynamics-cluster-agent-token-5dt8s kubernetes.io/service-account-token 4 30m appdynamics-infraviz-dockercfg-f9ncg kubernetes.io/dockercfg 1 30m appdynamics-infraviz-token-5qpfs kubernetes.io/service-account-token 4 30m appdynamics-operator-dockercfg-v9k7k kubernetes.io/dockercfg 1 30m appdynamics-operator-token-478lb kubernetes.io/service-account-token 4 30m builder-dockercfg-wkdnm kubernetes.io/dockercfg 1 30m builder-token-nvjmq kubernetes.io/service-account-token 4 30m cluster-agent-secret Opaque 1 30m default-dockercfg-6dn5t kubernetes.io/dockercfg 1 30m default-token-m7hkp kubernetes.io/service-account-token 4 30m deployer-dockercfg-25ws4 kubernetes.io/dockercfg 1 30m deployer-token-dd9s9 kubernetes.io/service-account-token 4 30m sh.helm.release.v1.abhi-cluster-agent.v1 helm.sh/release.v1 1 30m Step 4: Inspect the Content of a Secret To examine the contents of a specific secret, such as appdynamics-cluster-agent-dockercfg-zvglk , use the following command: kubectl get secret appdynamics-cluster-agent-dockercfg-zvglk -n appdynamics -o yaml The output will look something like below kubectl get secret appdynamics-cluster-agent-token-5dt8s -n appdynamics -o yaml apiVersion: v1 data: ca.crt: xxxxxxxx namespace: YXBwZHluYW1pY3M= service-ca.crt: xxxxxx== token: xxxxxxx kind: Secret metadata: annotations: kubernetes.io/created-by: openshift.io/create-dockercfg-secrets kubernetes.io/service-account.name: appdynamics-cluster-agent kubernetes.io/service-account.uid: 582c10bc-b3fb-4435-9adb-7c6dfb25c2ff creationTimestamp: "2024-09-03T17:42:20Z" name: appdynamics-cluster-agent-token-5dt8s namespace: appdynamics resourceVersion: "84672" uid: a7cc1cba-27e4-4b46-a33d-212614af9cad type: kubernetes.io/service-account-token Step 5: Decode and Inspect the Certificate To check the details of the certificate (e.g., ca.crt ), decode it, and then use openssl to view its content: echo "Content of ca.crt" | base64 - decode | openssl x509 -noout -text If the certificate is found and properly decoded, the output will resemble: Certificate: Data: Version: 3 (0x2) Serial Number: ... Signature Algorithm: sha256WithRSAEncryption Issuer: OU=openshift, CN=kube-apiserver-lb-signer ... Conclusion This guide has walked you through identifying the certificates used by a specific Kubernetes deployment, focusing on secrets. By following these steps, you can effectively manage and verify the certificates that secure your Kubernetes environments, ensuring the integrity and confidentiality of your applications. ... View more

How to Encrypt and Secure Your Machine Agent AccessKey Encryption and securing your credentials are of utmost importance now. The Machine Agent lets you configure the encryption of your AccessKey. Let's go through the steps.   1. Navigate to <Ma-Home> directory and Let's create a keyStore with below command: jre/bin/java -jar lib/secure-credential-store-tool-1.3.23.jar generate_ks -filename '/opt/appdynamics/secretKeyStore' -storepass 'MyCredentialStorePassword' This will create the keyStore for you. The output should look like: Successfully created and initialized new KeyStore file: /opt/appdynamics/secretKeyStore 2. Let's create a password to access this keyStore: jre/bin/java -jar lib/secure-credential-store-tool-1.3.23.jar obfuscate -plaintext 'MyCredentialStorePassword' The output should look like: s_-001-12-oRQaGjKDTRs=xxxxxxxxxxxxx= 3. Encrypt your AccessKey: jre/bin/java -jar lib/secure-credential-store-tool-1.3.23.jar encrypt -filename /opt/appdynamics/secretKeyStore -storepass 'MyCredentialStorePassword' -plaintext 'xxxxxx' The output should look like: -001-24-mEZsR+xxxxxxxxx==xxxxxxxxxxxx== Great, now let's edit <MA-Home>/conf/controller-info.xml file and edit the accessKey while also adding a few more parameters for the encryption <account-access-key>-001-24-mEZsR+nrScSXlewlZbTQgg==xxxxxxxxx==</account-access-key> <credential-store-password>s_-001-12-xxxxxxx=xxxxxx=</credential-store-password> <credential-store-filename>/opt/appdynamics/secretKeyStore</credential-store-filename> <use-encrypted-credentials>true</use-encrypted-credentials> Great work! You can deploy your Machine Agent now! ... View more