Splunk AppDynamics

Error 502 after starting Java Agent

CommunityUser
Splunk Employee
Splunk Employee

Hello there,

We're deploying a app dynamics environment here to test some Java applications and the machines itself (hence, using java_agent and machine_agent).

However, as soon as we start the Java agent, we get connection to the controller, and it returns an error 502, as seen in the log bellow:

[AD Thread Pool-Global0] 14 Jul 2017 11:03:32,826 ERROR ConfigurationChannel - HTTP Request failed: HTTP/1.1 502 Bad Gateway
[AD Thread Pool-Global0] 14 Jul 2017 11:03:32,826  WARN ConfigurationChannel - Could not connect to the controller/invalid response from controller, cannot get initialization information, controller host [fanatics3c.saas.appdynamics.com], port[443], exception [null]
[AD Thread Pool-Global0] 14 Jul 2017 11:03:32,826 ERROR ConfigurationChannel - Exception: NULL
[AD Thread-Metric Reporter1] 14 Jul 2017 11:04:08,316  WARN MetricHandler - Metric Reporter Queue full. Dropping metrics.
[qtp568531308-75] 14 Jul 2017 11:04:30,100  INFO TransactionMonitor - Creating new BTContext for thread Id [75]
[AD Thread Pool-Global1] 14 Jul 2017 11:04:32,827  INFO ConfigurationChannel - Detected node meta info: [Name:ProcessID, Value:5554, Name:appdynamics.ip.addresses, Value:fe80:0:0:0:840:71ff:fe11:d5b2%eth0,10.151.16.50, Name:supportsDevMode, Value:true]
[AD Thread Pool-Global1] 14 Jul 2017 11:04:32,827  INFO ConfigurationChannel - Sending Registration request with: Application Name [3c-lpt], Tier Name [3c-lpt-promotion], Node Name [ip-10-151-16-50], Host Name [ip-10-151-16-50] Node Unique Local ID [ip-10-151-16-50], Version [Server Agent v4.3.3.2 GA #15225 r${env.GIT_COMMIT} 2813-4.3.3.next-analytics]
[AD Thread Pool-Global1] 14 Jul 2017 11:04:33,579 ERROR ConfigurationChannel - HTTP Request failed: HTTP/1.1 502 Bad Gateway
[AD Thread Pool-Global1] 14 Jul 2017 11:04:33,580  WARN ConfigurationChannel - Could not connect to the controller/invalid response from controller, cannot get initialization information, controller host [fanatics3c.saas.appdynamics.com], port[443], exception [null]
[AD Thread Pool-Global1] 14 Jul 2017 11:04:33,580 ERROR ConfigurationChannel - Exception: NULL
[qtp568531308-78] 14 Jul 2017 11:04:50,807  INFO TransactionMonitor - Creating new BTContext for thread Id [78]
[AD Thread-Metric Reporter0] 14 Jul 2017 11:05:08,316  WARN MetricHandler - Metric Reporter Queue full. Dropping metrics.
[AD Thread Pool-Global1] 14 Jul 2017 11:05:29,650  INFO ControllerTimeSkewHandler - Controller Time Skew Handler Run Aborted - Skew Check is Disabled
[AD Thread Pool-Global1] 14 Jul 2017 11:05:33,581  INFO ConfigurationChannel - Detected node meta info: [Name:ProcessID, Value:5554, Name:appdynamics.ip.addresses, Value:fe80:0:0:0:840:71ff:fe11:d5b2%eth0,10.151.16.50, Name:supportsDevMode, Value:true]
[AD Thread Pool-Global1] 14 Jul 2017 11:05:33,581  INFO ConfigurationChannel - Sending Registration request with: Application Name [3c-lpt], Tier Name [3c-lpt-promotion], Node Name [ip-10-151-16-50], Host Name [ip-10-151-16-50] Node Unique Local ID [ip-10-151-16-50], Version [Server Agent v4.3.3.2 GA #15225 r${env.GIT_COMMIT} 2813-4.3.3.next-analytics]
[AD Thread Pool-Global1] 14 Jul 2017 11:05:34,275 ERROR ConfigurationChannel - HTTP Request failed: HTTP/1.1 502 Bad Gateway
[AD Thread Pool-Global1] 14 Jul 2017 11:05:34,275  WARN ConfigurationChannel - Could not connect to the controller/invalid response from controller, cannot get initialization information, controller host [fanatics3c.saas.appdynamics.com], port[443], exception [null]
[AD Thread Pool-Global1] 14 Jul 2017 11:05:34,275 ERROR ConfigurationChannel - Exception: NULL
[qtp568531308-80] 14 Jul 2017 11:05:40,909  INFO TransactionMonitor - Creating new BTContext for thread Id [80]

Keep on mind that other agents (like Db and Machine agent) works well. We're not using the dbagent today, but we already tested it, and seems to be ok. The MachineAgent is reporting back right to the server and client, and show no errors.

Our controller-info.xml file is:

<controller-info>
    <controller-host>[REDACTED].saas.appdynamics.com</controller-host>
    <controller-port>443</controller-port>
    <controller-ssl-enabled>true</controller-ssl-enabled>

    <account-name>[REDACTED]</account-name>
    <account-access-key>[REDACTED]</account-access-key>

    <application-name>3c-lpt</application-name>
    <tier-name>3c-lpt-promotion</tier-name>
    <node-name>ip-10-151-16-50</node-name>
</controller-info>
Labels (1)
0 Karma

Rajesh_Putta
Communicator

Hi,

Could you please send us zipped agent logs to validate few things.

Thanks

Rajesh

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Good morning Rajesh,

I found the solution last Friday. our issue here was provoked due to a new behavior of the Java Agent, which uses a versioning system since the very first 4.3.x agent. For some reason, the local default configuration hadn't some parameters comented out (like SSL connection to controller) conflicting with our global controller-info.xml file. 

The solution was just replace the global config file for an additional copy of our generated controller-info.xml

0 Karma

Rajesh_Putta
Communicator

Glad to hear this good news. Please let us know if we can solve this one,

Thanks

Rajesh

0 Karma
Get Updates on the Splunk Community!

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL  The Splunk AI Assistant for SPL ...

Buttercup Games: Further Dashboarding Techniques (Part 5)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Customers Increasingly Choose Splunk for Observability

For the second year in a row, Splunk was recognized as a Leader in the 2024 Gartner® Magic Quadrant™ for ...