Splunk AppDynamics

AppDynamics Smart Agent Management with GitHub Actions

aleccham
Explorer

AppDynamics Smart Agent Management with GitHub Actions

Automated deployment and lifecycle management of AppDynamics Smart Agent across multiple EC2 hosts using GitHub Actions with a self-hosted runner.

https://github.com/chambear2809/github-actions-lab/tree/main

🎯 Overview

This lab demonstrates how to use GitHub Actions to manage AppDynamics Smart Agent and various AppDynamics agents (Node, Machine, DB, Java) across multiple Ubuntu EC2 instances within a single AWS VPC. The project includes 11 automated workflows covering the complete agent lifecycle.

Key Features:

  • 🚀 Parallel Deployment - Deploy to multiple hosts simultaneously
  • 🔄 Complete Lifecycle Management - Install, uninstall, stop, and clean agents
  • 🏗 Infrastructure as Code - All workflows version-controlled
  • 🔐 Secure - SSH key-based authentication, private VPC networking
  • 📈 Massively Scalable - Deploy to thousands of hosts with automatic batching
  • 🎛 Self-hosted Runner - Executes within your AWS VPC

📊 Architecture

All infrastructure runs in a single AWS VPC with a shared security group. The self-hosted runner communicates with target hosts via private IPs.

View detailed architecture diagrams →

🚀 Quick Start

Prerequisites

  • AWS VPC with EC2 instances (Ubuntu)
  • Self-hosted GitHub Actions runner in the same VPC
  • SSH key pair for authentication
  • AppDynamics Smart Agent package and config

1️⃣ Clone and Configure

git clone https://github.com/chambear2809/github-actions-lab.git
cd github-actions-lab
 

2️⃣ Set Up GitHub Secrets

Navigate to: Settings → Secrets and variables → Actions

Required Secret:

  • SSH_PRIVATE_KEY - Your SSH private key (PEM format)

3️⃣ Set Up GitHub Variables

Navigate to: Settings → Secrets and variables → Actions → Variables

Required Variable:

  • DEPLOYMENT_HOSTS - Target host IPs (one per line)
    172.31.1.243
    172.31.1.48
    172.31.1.5
     

Optional Variables:

  • SMARTAGENT_USER - User for Smart Agent service (e.g., appdynamics)
  • SMARTAGENT_GROUP - Group for Smart Agent service (e.g., appdynamics)

4️⃣ Deploy!

Via GitHub UI:

  1. Go to Actions tab
  2. Select "Deploy Smart Agent"
  3. Click "Run workflow"
  4. Optionally adjust batch size (default: 256)
  5. Click "Run workflow"

Via GitHub CLI:

gh workflow run "Deploy Smart Agent" --repo chambear2809/github-actions-lab

# With custom batch size
gh workflow run "Deploy Smart Agent" --repo chambear2809/github-actions-lab -f batch_size=128
 

📋 Available Workflows

Deployment (1 workflow)

Workflow Description Scale Trigger

Deploy Smart Agent (Batched)Installs Smart Agent and starts serviceAnyManual only

Agent Installation (4 batched workflows)

Workflow Command Scale Trigger

Install Node Agent (Batched)smartagentctl install nodeAnyManual only
Install Machine Agent (Batched)smartagentctl install machineAnyManual only
Install DB Agent (Batched)smartagentctl install dbAnyManual only
Install Java Agent (Batched)smartagentctl install javaAnyManual only

Agent Uninstallation (4 batched workflows)

Workflow Command Scale Trigger

Uninstall Node Agent (Batched)smartagentctl uninstall nodeAnyManual only
Uninstall Machine Agent (Batched)smartagentctl uninstall machineAnyManual only
Uninstall DB Agent (Batched)smartagentctl uninstall dbAnyManual only
Uninstall Java Agent (Batched)smartagentctl uninstall javaAnyManual only

Smart Agent Management (2 batched workflows)

Workflow Description Scale Trigger

Stop and Clean Smart Agent (Batched)Stops service and purges dataAnyManual only
Cleanup All Agents (Batched)Deletes /opt/appdynamics directoryAnyManual only

Total: 11 workflows - All batched workflows support configurable batch sizes (default: 256)

📚 Documentation

🛠️ How It Works

  1. Developer pushes code or manually triggers a workflow
  2. GitHub Actions receives the event and assigns job to self-hosted runner
  3. Runner loads target hosts from GitHub variables
  4. Parallel Execution - Runner SSHs into each target host simultaneously
  5. Commands Execute - Install/uninstall/stop/clean operations run on each host
  6. Results Reported - Success/failure status sent back to GitHub

🔐 Security

  • Private Network - All communication via VPC private IPs
  • SSH Keys - Stored securely as GitHub secrets
  • No Public Access - Target hosts don't need public IPs
  • Security Group - Restricts SSH access to runner only

📈 Scaling

All workflows use automatic batching to support any number of hosts:

How It Works

  • Automatic batching - Splits hosts into groups of 256 (configurable)
  • Sequential batch processing - Avoids overwhelming runner resources
  • Parallel within batch - Each batch processes all hosts simultaneously
  • Works at any scale - 1 host to thousands

Batching Strategy

  1. Splits your host list into manageable batches
  2. Processes each batch as a separate job sequentially
  3. Deploys to all hosts within each batch in parallel using background processes

Examples:

  • 10 hosts = 1 batch, all deploy in parallel
  • 500 hosts = 2 batches × 256 hosts
  • 1,000 hosts = 4 batches × 256 hosts
  • 5,000 hosts = 20 batches × 256 hosts

🤝 Contributing

This is a lab/demo project. Feel free to fork and adapt for your own use cases!

🔗 Links

Labels (2)
0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...