What are the steps for installing OTel Collector as a sidecar in ECS Fargate to monitor applications?
This article provides step-by-step instructions and sample configuration files you can use to install an OTel collector as a sidecar in ECS Fargate, so you can monitor applications.
It includes two alternatives for configuring your application to use OpenTelemetry: (a) by modifying your existing application image, or (b) by injecting the OTel Java agent into your container.
In this article... Installation instructions | Resources
Installation instructions
Go to AppDynamics Cloud Tenant UI > Configure > Kubernetes & APM and generate the agent credentials. Download the file or copy the snippet that includes clientId, clientSecret, endpoint, and tokenUrl.
Create a collector-values.yaml file. Replace content as follows:
REPLACE:
WITH:
<<client-id>> <<client-secret>> <<token-url>>
Replace with the values from step 1.
<<tenant-url>>
Replace with the actual tenant URL
processors:
batch:
timeout: 10s
send_batch_size: 8192
receivers:
otlp:
protocols:
grpc:
http:
exporters:
logging:
#loglevel: debug
verbosity: detailed
otlphttp:
logs_endpoint: https://<<tenant-url>>/data/v1/logs
metrics_endpoint: https://<<tenant-url>>/data/v1/metrics
traces_endpoint: https://<<tenant-url>>/data/v1/trace
auth:
authenticator: oauth2client
extensions:
oauth2client:
client_id: <<client-id>>
client_secret: <<client-secret>>
token_url: <<token-url>>
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp, logging]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
logs:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
extensions: [oauth2client]
telemetry:
logs:
level: "debug"
In the same directory as the collector-values.yaml file, create an otelcollector-dockerfile file with the following content:
# Sample Dockerfile for the OpenTelemetry Contrib Collector SideCar Container.
# This is provided for illustration purposes only, for full details
# please consult the product documentation: https://docs.appdynamics.com/
FROM debian
# Install Required Packages
RUN apt-get update && apt-get -y install curl && apt-get -y install openssl && apt-get -y install bash && apt-get clean
# Create a work directory to copy the Otel Collector artifacts
ENV APP_HOME /opt/otel
RUN mkdir -p ${APP_HOME}
WORKDIR ${APP_HOME}
# Download and extract Contrib version of Otel Collector artifacts to the work directory
RUN curl -L0 https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.75.0/otelcol-contrib_0.75.0_linux_386.tar.gz --output otelcol-contrib_0.75.0_linux_386.tar.gz
RUN tar -xvf otelcol-contrib_0.75.0_linux_386.tar.gz
RUN rm -rf otelcol-contrib_0.75.0_linux_386.tar.gz
# Copy the config file for Otel Collector to work directory
COPY collector-values.yaml ${APP_HOME}
# Expose the /opt/otel directory as a shared volume
#VOLUME ["/opt/otel"]
# Expose ports
EXPOSE 4317 4318 8888 8889 13133 55679
CMD ${APP_HOME}/otelcol-contrib --config ${APP_HOME}/collector-values.yaml
Build, tag, and push the docker image. Then, replace <repository-url> as shown below. docker build -t otel-collector-contrib:latest --no-cache -f otelcollector-dockerfile .
docker tag otel-collector-contrib:latest <repository-url>/otel-collector-contrib:latest
docker push <repository-url>/otel-collector-contrib:latest
Configure your application to use OpenTelemetry using one of two ways: • Modify your existing application image • Inject OpenTelemetry Java Agent into your application as an init container
Option 1 | Configure your app to use OTel by modifying the existing application image
Modify your existing application image and add javaagent:path/to/opentelemetry-javaagent.jar to your JVM’s startup arguments.
Download opentelemetry-javaagent.jar from Releases of the opentelemetry-java-instrumentation repo. The JAR file contains the agent and all automatic instrumentation packages.
Add the JAR file to your application build file. See the build file below for reference. FROM openjdk:17-jdk-slim-buster
RUN apt-get -y update && apt-get -y install curl
LABEL version="1.0.1"
RUN mkdir -p /opt/app/otelincsaas
WORKDIR /opt/app/otelincsaas
# Copy Jar file.
COPY otelincsaas/ /opt/app/otelincsaas/
COPY otelincsaas/opentelemetry-javaagent.jar /opt/app/otelincsaas/
RUN chmod -R 755 /opt/app/otelincsaas
ENTRYPOINT ["java", "-javaagent:/opt/app/otelincsaas/opentelemetry-javaagent.jar", "-jar","otel-in-csaas-1.0.0.jar"]
Add -javaagent:<path-to-opentelmetry-jar>/opentelemetry-javaagent.jar to your startup command.
Next step
Option 2 | Configure your app to use OTel by injecting the (OTel) Java Agent into your application as an init container
Create a Docker file. otelagent-dockerfile
# Sample Dockerfile for the OpenTelemetry Agent SideCar Container.
# This is provided for illustration purposes only, for full details
# please consult the product documentation: https://docs.appdynamics.com/
FROM debian
# Install Required Packages
RUN apt-get update && apt-get -y install curl && apt-get -y install openssl && apt-get -y install bash && apt-get clean
# Create a work directory to copy the Otel Agent artifacts
ENV APP_HOME /opt/otel
RUN mkdir -p ${APP_HOME}
WORKDIR ${APP_HOME}
# Download and extract Otel Agent artifacts to the work directory
RUN curl -L0 https://github.com/aws-observability/aws-otel-java-instrumentation/releases/download/v1.24.0/aws-opentelemetry-agent.jar --output aws-opentelemetry-agent.jar
# Expose the /opt/otel directory as a shared volume
VOLUME ["/opt/otel"]
CMD tail -f /dev/null
Build, tag, and push the Docker image. Replace <repository-url> accordingly. Build, Tag, and Push
docker build -t otel-java-agent:latest --no-cache -f otelagent-dockerfile .
docker tag otel-java-agent:latest <repository-url>/otel-java-agent:latest
docker push <repository-url>/otel-java-agent:latest
At this stage, we have images for the:
application
otel-collector
otel-agent
Create a task definition in your ECS Fargate cluster and deploy the images. A
ADD A CONTAINER FOR:
otel-java-agent
Name: otel-java-agent
Image URL: <repository-url>/otel-java-agent:latest
Essential Container: No
otel-collector-contrib
Name: otel-collector-contrib
Image URL: <repository-url>/otel-collector-contrib:latest
Essential Container: Yes
<your application>
Name: <application container name here>
Image URL: Application Image
Essential Container: Yes
Set the other details according to your Application.
Set the environment variables in the following table, along with others, if any.
Environment Variables Template: KEY
VALUE TYPE
VALUE
OTEL_SERVICE_NAME
value
Name of the Service
OTEL_RESOURCE_ATTRIBUTES
value
service.namespace=name of the application
JAVA_TOOL_OPTIONS
value
-javaagent:/opt/otel/aws-opentelemetry-agent.jar
OTEL_LOGS_EXPORTER
value
otlp
OTEL_TRACES_EXPORTER
value
otlp
OTEL_EXPORTER_OTLP_INSECURE
value
true
OTEL_PROPAGATORS
value
tracecontext,baggage,b3
Edit the task definition in JSON format. For your application container, add the following properties and create a new task definition. "volumesFrom": [
{
"sourceContainer": "otel-java-agent"
}
],
"dependsOn": [
{
"containerName": "otel-java-agent",
"condition": "START"
}
]
Deploy the service.
Resources
The opentelemetry-javaagent.jar is available in "Releases", opentelemetry-java-instrumentation
... View more