topic Re: Monitoring an application that uses mainly WebSocket communication in Splunk AppDynamics

Monitoring an application that uses mainly WebSocket communication

CommunityUser — Tue, 13 Mar 2018 00:21:25 GMT

Our application is a cloud platform that gives users access to embedded systems and IoT devices. Whereas we do have a small HTTP-based web API, the majority of communication and interaction happens via Web Socket communication on three different ports. On some of the ports the higher-level transmission protocol is JSON-RPC 2.0, other ports use a number of custom binary protocols.

We use the Netty 4.1 framework to implement WebSocket communication.

Little to no traffic happens over HTTP.

AppDynamics has been able to identify some basic business transactions, mainly centered around GWT, which is one of the mechanisms we use for front-end communication. However, for device communication AppD has not been able to auto-detect meaningful endpoints.

So, my detailed questions are:

What is a good way to set-up monitoring for the WebSocket communication that has not been autodetected?
How do we set up a health check that, e.g., sends a particular JSON-RPC message on the Web Socket and measures the time until we receive a response?

Re: Monitoring an application that uses mainly WebSocket communication

CommunityUser — Tue, 16 Apr 2019 04:07:25 GMT

I have the same issue. Did you get any answer for this question?

Re: Monitoring an application that uses mainly WebSocket communication

CommunityUser — Tue, 16 Apr 2019 17:28:17 GMT

No, we never got this resolved. Our conclusion was that AppD's out-of-the-box support for ws:/wss: communication is very limited. It probably would be possible to set it up, but it seems that a lot of manual configuration would be required.

Since our own software (which we were trying to monitor with AppD) itself happens to be a sensor data collection and analysis system (though primarily for geophysical sensors) we ended up implementing our own "self-monitoring" system by introducing synthetic sensors that were implemented via JMX, etc.

Re: Monitoring an application that uses mainly WebSocket communication

CommunityUser — Wed, 08 May 2019 20:19:49 GMT

We're evaluating the product for our NodeJS Express/Socket.io app and not seeing anything reporting in the dashboard for websocket events.

We are also NOT seeing any remote HTTP calls our app makes to external HTTP REST endpoints.

The only thing we're seeing in the "Remote Services" tab is our connection to Redis.

Although they advertise HTTP and Socket.io monitoring, it doesn't look like it's working very well.

Re: Monitoring an application that uses mainly WebSocket communication

Peter_Holditch — Thu, 09 May 2019 10:40:14 GMT

Conceptually, the core construct that AppDynamics monitors is event based communication, where the processing of each message is one Business Transaction.

Being stream based, rather than message based, raw web sockets do not fit well into this model. If your application operates a higher level message-based protocol over websockets where a complete message is sent over the socket and received, processed and replied to on the server, then the server side activity should be modelled as a Business Transaction, and will be tracked as such by AppDynamics.

Business Transactions must start from an entry point the agent knows about. The agents automatically detect some entry-points by virtue of the transaction logic being placed in a well-known container (e.g. within a servlet for java HTTP). Business Transactions originating at points in code that are not known to the AppDynamics require custom configuration to be recognised (a POJO entry point, in the case of java, or using the agent API in the case of the Node.js agent).

Backend activity (e.g. your external REST calls) will only be tracked if it is in the context of a Business Transaction (and made through a client that is either instrumented out of the box, or via configuration)

To my knowledge, none of our agents support out of the box instrumentation for socket.io; a quick look suggests to me that is is event based, so if you were to instrument socker.emit() as an exit and socket.on() as a Business Transaction entry, I think you would see what you are expecting.

If the messages are passing between server-side components and you additionally arrange for the AppDynamics correlation header to be passed in the message payload then AppDynamics would also show you the end to end correlation.

I hope that explanation helps.

Warm regards,

Peter