Splunk AppDynamics

Service Fabric Project

CommunityUser
Splunk Employee
Splunk Employee

Hi,

I was waiting for a long time for version 4.4 since we use a .Net Core Service Fabric application in Azure with a bunch of Stateless and Stateful Services running across multiple nodes.

Now I have followed this documentation https://docs.appdynamics.com/display/PRO44/Install+AppDynamics+for+Azure+Service+Fabric 

and I also read a bit here
 https://docs.appdynamics.com/display/PRO44/PDF+Documentation?preview=%2F43910190%2F45484022%2F4.4.1_...

and I can't seem to make it work. The documentation seems a bit ambiguous to say the least. I am in need of advice in getting appdynamics up and running. I might be doing something wrong, but don't know what. I have added the nuget package via Visual Studio, and even got this nice little pop-up asking me for info about our controller, which I filled in for every service.

One thing i noticed is if I rename AppDynamicsConfig.json, the build breaks.
Secondly, I might not have placed the dll files where I should have. Should they be where the service manifest is? Visual Studio creates a folder called AppDynamics where all the dll and json files are. (under each service)

Any help will be greatly appreciated.

Labels (1)
0 Karma
1 Solution

CommunityUser
Splunk Employee
Splunk Employee

2018-03-19 05:57:56.1361 4140 Visma.Cfa.Asf.ConcurrentActor 1 3 Error ConfigurationChannel Exception: The underlying connection was closed: The connection was closed unexpectedly. Exception: System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
at System.Net.HttpWebRequest.GetResponse()
at AppDynamics.Shared.Communication.hp.rt(HttpWebRequest )
at AppDynamics.Shared.Communication.hp.ym(Byte[] , String )
at com.appdynamics.ee.rest.controller.request.ja.gua()


2018-03-19 05:58:05.0455 2964 Visma.Cfa.Asf.Configuration.Service 1 3 Warn ConfigurationChannel Could not connect to the controller/invalid response from controller, cannot get initialization information, controller host [vismadev.saas.appdynamics.com], port[443], exception [System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
at System.Net.HttpWebRequest.GetResponse()
at AppDynamics.Shared.Communication.hp.rt(HttpWebRequest )
at AppDynamics.Shared.Communication.hp.ym(Byte[] , String )
at com.appdynamics.ee.rest.controller.request.ja.gua()]

Above are examples of Errors I get in the Appdynamics log(WarnLog) on the Service Fabric Nodes/VMs.

I have set it up exactly as you described and as you can see in the screenshots. I have also tried:

telnet vismadev.saas.appdynamics.com 443 
from the VM and it does work. I suspect SSL not working or some firewall issue, or maybe I need an open port that I don't know about. Do you have any advice on what to do next?

View solution in original post

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi @Anonymous,

I talked with our .NET developer and it appears that you installed our NuGet package in the project using Visual Studio, which is not supported yet. Please download the package manually using a browser, then extract it to a folder as a zip archive. After that, follow the instructions for Service Fabric, which includes manually copying/pasting the files.

I'll update the docs to reflect this.

Thanks,

Cody

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi,

I see. It was wrong to assume. I removed everything via Visual Studio and am adding everything manually, BUT I still have some difficulty understanding which folder is which and where I should actually place the files. I have attached a screenshot of ONE stateful service and the folder structure. Which one is the "code package folder that contains the executable"(or in other words where should I place the .dll files and such)? I also had an error, previously, when renaming the "AppDynamicsConfig.json" file. I think Appdynamics was still looking for a file with that name. How do we handle that?

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi @Anonymous,

I'm assuming that the code package folder that contains the executable you want to run is the executable of the application that you want to monitor using AppDynamics, but let me check on that.

If renaming the .json file breaks the build, can you rename it back?

Let me do some research on my end and will get back to you.

Cody

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

The code package folder that contains the executable is the executable of the application you want to monitor.

Have you tried renaming your .json file back to the original name?

Cody

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi,

Renaming the .json back to the original name fixed the build but the controller still isn't receiving anything. How can one debug this? Are there appdynamics logs kept somewhere if I run the application locally?

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi @Anonymous sorry for the delay in my reply. I'm trying to get you some additional help. Did anyone else reach out to you?

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hello. I appreciate it. No, not yet. 🙂

0 Karma

Raunak_Mohanty
Builder

Hi Tudor,

  We are trying it out internally and will update you soon. Apologies for the delay here.

Thanks,

Raunak

Kartikay_Tripat
Path Finder

Hi Tudor, 

We have tested the steps for Installing service fabric as given in - https://docs.appdynamics.com/display/PRO44/Install+AppDynamics+for+Azure+Service+Fabric

We were able to make the instrumentation work and here is what we followed -

  • We Manually downloaded the NugetPackage from https://www.nuget.org/api/v2/package/AppDynamics.WindowsAzure/4.4.1
  • We unzipped it in a folder on the machine. 
  • We copied AppDynamics.Agent.dllAppDynamics.Profiler_x64.dllAppDynamicsAgentLog.configAppDynamicsConfig.json from <nuget_package>\content\AppDynamics and added in the Visual studio solution of each of the Service Project at the top level, not under any folder. 
  • Right Click on each of these 4 newly added files in Visual studio for each service project and make sure under properties Build Action: Content and Copy to Output Directory: Copy Always is selected.
  • Rename AppDynamicsConfig.json to <executable_name>.AppDynamicsConfig.json in each project.
  • Also modify the ServiceManifest.xml in each of the project and add following  - 
<CodePackage ...>
  <EntryPoint>
    ...
  </EntryPoint>
  <EnvironmentVariables>
    <EnvironmentVariable Name="COR_ENABLE_PROFILING" Value="1" />
    <EnvironmentVariable Name="COR_PROFILER" Value="{39AEABC1-56A5-405F-B8E7-C3668490DB4A}" />
    <EnvironmentVariable Name="COR_PROFILER_PATH" Value="AppDynamics.Profiler_x64.dll" />
   </EnvironmentVariables>
</CodePackage>
  • Update the <executable_name>.AppDynamicsConfig.json in each service project, with following  - 
{
"controller":
{
"host":"ControllerHost",
"port":ControllerPort,
"account":"ControllerAccount",
"password":"AccessKey"
},
"application":
{
"name":"NameOfTheControllerApplication"
}
}
  • Publish the Services to the Cluster from Visual Studio, it will publishes the added new files as content of the deployed package.
  • On applying the load on the applications, we started seeing the tiers getting registered for each hosted service in controller.
  • Agent logs were getting created under D:\LogFiles\AppDynamics on the Cluster VMs. 

Note: As per your screenshot looks like you have added the nuget package by the package manager in the solution, that is not needed. Which creates a folder Appdynamics in the project and adds all the files in it. The steps shared above is needed to be performed manually after downloading and extracting the nuget package on the machine.

Attached screenshot of the my test project which I created for testing the behaviour.

CommunityUser
Splunk Employee
Splunk Employee

2018-03-19 05:57:56.1361 4140 Visma.Cfa.Asf.ConcurrentActor 1 3 Error ConfigurationChannel Exception: The underlying connection was closed: The connection was closed unexpectedly. Exception: System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
at System.Net.HttpWebRequest.GetResponse()
at AppDynamics.Shared.Communication.hp.rt(HttpWebRequest )
at AppDynamics.Shared.Communication.hp.ym(Byte[] , String )
at com.appdynamics.ee.rest.controller.request.ja.gua()


2018-03-19 05:58:05.0455 2964 Visma.Cfa.Asf.Configuration.Service 1 3 Warn ConfigurationChannel Could not connect to the controller/invalid response from controller, cannot get initialization information, controller host [vismadev.saas.appdynamics.com], port[443], exception [System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
at System.Net.HttpWebRequest.GetResponse()
at AppDynamics.Shared.Communication.hp.rt(HttpWebRequest )
at AppDynamics.Shared.Communication.hp.ym(Byte[] , String )
at com.appdynamics.ee.rest.controller.request.ja.gua()]

Above are examples of Errors I get in the Appdynamics log(WarnLog) on the Service Fabric Nodes/VMs.

I have set it up exactly as you described and as you can see in the screenshots. I have also tried:

telnet vismadev.saas.appdynamics.com 443 
from the VM and it does work. I suspect SSL not working or some firewall issue, or maybe I need an open port that I don't know about. Do you have any advice on what to do next?

0 Karma

Raunak_Mohanty
Builder

Hi,

  Can you try using port 80 which is a non ssl port and verify if the connection goes through? After changing the port info recycle the process.

Good thing is that Agent is able to hook into the process.

Thanks,

Raunak

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

It worked on port 80, thank you for that, but how do I find out why port 443 is not working?

0 Karma

Raunak_Mohanty
Builder

Hi Tudor,

  Can you double check that both ssl and enable_tls12 is set to true like shown below?

<controller host="mycontroller.mycompany.com" port="443" ssl="true" enable_tls12="true" 

If above does not work, can you also double check if port 443 is not blocked in anyway.

Thanks,

Raunak

CommunityUser
Splunk Employee
Splunk Employee

Hi,

It indeed worked as you suggested. I noticed a few issues I would kindly ask for your help with:

1. If you have the service fabric SDK installed you can debug your application locally.

I see appdynamics detected each Service where I put the .DLLs in as a Tier, I guess, and each Node it's running on both in Azure and locally on my machine. The problem is I have 5 Nodes on my PC(simulated) and 3 production Nodes in Azure. I see all 8 of them in Appdynamics. Could we somehow isolate the local nodes and ignore them? Same goes for accessing local databases versus Azure databases.

2. I copied some dll all around our solution and that made the package size go up quite alot. Couldn't we just point Appdynamics in each service towards the same AppDynamics.Agent.dll and profiles file?

3. We know for sure that we have multiple services being used, but only 3 of them show up in Appdynamics. I know that appdynamics detects load and only shows that, but we are pretty sure we are using all services together at all times. I will still have to triplecheck this, but it is my impression that only 3 services actually send data to the Controller.

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi @Anonymous -- I wrote that documentation with some help from a developer. Let me ping him and see if he might know what the issue is. Sorry you're having trouble -- let's get you going and then fix the docs, if needed.

Cody

0 Karma
Get Updates on the Splunk Community!

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL  The Splunk AI Assistant for SPL ...

Buttercup Games: Further Dashboarding Techniques (Part 5)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Customers Increasingly Choose Splunk for Observability

For the second year in a row, Splunk was recognized as a Leader in the 2024 Gartner® Magic Quadrant™ for ...