topic Configuring Java agent for multiple replicas or instances in AppD Archive

Configuring Java agent for multiple replicas or instances

Akshay_Nimbal — Fri, 23 Feb 2024 05:30:06 GMT

We have a kubernetes cluster and each application is running as multiple replicas as in multiple pods. What's the best way to track the Java application from multiple replicas? Is it right to treat each replica as a node if yes, this would be an issue when autoscaling happens as the new replica in upscaling would be assigned a node which is not yet created in AppD console. Or do we have any other option on AppD console to capture separate metrics for both the replicas with or without the same agent.

Re: Configuring Java agent for multiple replicas or instances

Paula_Semaan1 — Mon, 04 Mar 2024 16:41:08 GMT

Hi Akshay,

yes you are right, each replica would be treated as a node.

We'd recommend deploying the cluster agent and leveraging it to auto-deploy the java agent and instrument your java application:

https://docs.appdynamics.com/appd/24.x/24.2/en/infrastructure-visibility/monitor-kubernetes-with-the-cluster-agent/auto-instrument-applications-with-the-cluster-agent

There are a few steps in the link above related to the tiers and nodes naming and management:

Pre-requisite:
- Ensure that you have specified the app, the tier, and the node name tuple to be unique across all the Kubernetes instances. If the name is not unique, the nodes may not report properly. This is applicable to those agents on which app, tier, and node uniqueness is required such as Java and Node.js Agents.

...

At the "Enable Auto-Instrumentation for the Cluster Agent" step:
- Remove any deleted pods from the Tiers & Nodes Controller Dashboard. Then, re-create the cluster-agent-secret that was created in Install the Cluster Agent to include api-user:
```
kubectl -n appdynamics delete secret cluster-agent-secret
kubectl -n appdynamics create secret generic cluster-agent-secret --from-literal=controller-key=<access-key> --from-literal=api-user="<username>@<customer>:<password>"
```
  CODE
  Set the api-user value to a local user from the Controller with the Server Monitoring Administrator role. Cluster Agent uses the api-user to mark the associated node in the Controller as historical on pod deletion.
  
  ...
AppDynamics Tier Name Strategies
- The Controller's Application Dashboard > Tiers and Nodes provides two tier name strategies. Select a strategy by assigning the tierNameStrategy property to one of these values:
  - manual: Use the tierName parameters in the cluster-agent.yaml file to set the tier name.
  - label: Use a label from the workload's spec as the tier name

Hope this helps, let us know please if you need further information.

Re: Configuring Java agent for multiple replicas or instances

Akshay_Nimbal — Tue, 05 Mar 2024 05:42:31 GMT

Thank you @Paula.Semaan for the response.

After some more digging, we went with -Dappdynamics.agent.reuse.nodeName=true -Dappdynamics.agent.reuse.nodeName.prefix=${HOSTNAME}

So, this way we could dynamically register new pods/instances/replicas coming in our Kubernetes cluster and this node would be registered in the AppD console as the pod name as shown below.

So this handled upscaling/downscaling and updating in the console.

1. Do you see any downsides in this as we only wanted to use the Java agent?

2. We have a multi-region deployment of our applications, can you suggest any better organization structure for our apps and nodes in this case? We were thinking of using AppName -> Tier as region ( us-east-1 is one tier, eu-west-1 is another tier, etc ) -> Nodes as Kubernetes pods under that tier/region.

Thank you

Re: Configuring Java agent for multiple replicas or instances

Paula_Semaan1 — Tue, 05 Mar 2024 13:12:16 GMT

Hi @Akshay.Nimbal

yes, this is a great usage of the properties -Dappdynamics.agent.reuse.nodeName=true -Dappdynamics.agent.reuse.nodeName.prefix=${HOSTNAME}, because they'll ensure a continuity of metric reporting for a node when the pod changes.

1. No I don't see any downside for this solution, it's a very good one

2. In theory, we'd recommend naming tiers based on the business purpose / service they fulfil.

For example in this picture, a Checkout business transaction:

Is being handled at the start by the tier "Ecommerce-services" (which has 40 nodes) which aim is to be the first level of the request processing and to commit the transaction in the database
Then the Inventory Services tier (with 4 nodes), which aim is to check the availability of the item, update the counter accordingly, etc, is called in order to make an API call to the main supplier
Then the order is put in a queue that is processed by the "Order Processing Services" tier which aim is to process orders being placed, and that makes a call to another remote service.

Splitting the tiers by region, even though they have common business purposes across these regions, would lead to different business transactions for "Checkout" for example, each originating from a regional tier, for example:
. A Checkout transaction (with its own metrics and snapshots) originating from "us-east-1"

. Another Checkout transaction (with its own metrics and snapshots) originating from eu-west-1

etc.

The downsides are that:

1. You wouldn't be able to aggregate automatically the metrics of the business transaction and the tiers across the different regions.

2. This could increase very easily the number of business transactions close to the default limit of 200, which is not recommended.

Here are the best practices around an application organisation in AppDynamics (application name, tiers names, nodes names, etc):
https://docs.appdynamics.com/appd/24.x/latest/en/application-monitoring/overview-of-application-monitoring

Here is a great article on how to define business transactions (which is very related to the choice of tiers organisation):

https://www.appdynamics.com/blog/product/how-to-identify-impactful-business-transactions-in-appdynamics

Happy to discuss further over a call!