topic Re: `NPM install` integrity check failure on Linux in Splunk AppDynamics

`NPM install` integrity check failure on Linux

CommunityUser — Tue, 21 Nov 2017 16:31:57 GMT

integrity checksum failed when using sha1: wanted sha1-SdstUAFQuOSM8Wed69GwsB/wLx8= but got sha1-bocDAWWpWcjMh/14FCG4/eDNOZM=

That is the exact error I get.

That checksum is from my package-lock.json file

"appdynamics-zmq": {
"version": "http://packages.appdynamics.com/nodejs/4.4.1.0/appdynamics-zmq.tgz",
"integrity": "sha1-SdstUAFQuOSM8Wed69GwsB/wLx8="
 }

That chechsum was generated on OSX. BUT, it's a file. Its hash should be the same on any platform because it's the same file being downloaded.

From the NPM documentation (source: https://docs.npmjs.com/files/package-lock.json)

> For remote tarball sources this is an integrity based on a SHA512 of the file.

Therefore the only explaination that I can think of is that the appDynamics server is serving a different file to my Linux machine than it serves to my OSX machine...

Re: `NPM install` integrity check failure on Linux

Ray_Waldock — Wed, 22 Nov 2017 22:41:19 GMT

Hi,

packages.appdynamics.com serves the file based on the provided user-agent string.

For additional information see: https://community.appdynamics.com/t5/Knowledge-Base/Advanced-NPM-Guide-for-the-AppDynamics-Node-js-Agent/ta-p/26743

Re: `NPM install` integrity check failure on Linux

CommunityUser — Thu, 23 Nov 2017 19:49:42 GMT

That would explain it. But then the issue is that the server doing that is incompatible with using a package-lock.json file and deploying on a different platform than the one that a developer develops on... The integrity check will fail every time...

Re: `NPM install` integrity check failure on Linux

Kyle_Furlong — Mon, 27 Nov 2017 18:11:36 GMT

Hi Marc,

That's correct, we currently do not support lockfiles with our install workflow, due to the nature of our native dependencies. One way to deploy AppD if you want to keep using lockfiles would be to install our package once globally for each instrumented machine. This would make the agent available to every app on the machine, without having to worry about whether you copied the right one over from dev/build.

Regards,

Kyle

Re: `NPM install` integrity check failure on Linux

CommunityUser — Mon, 16 Jul 2018 15:52:32 GMT

Hi Kyle

I realise this is an old post, but this issue is causing us all sorts of problems.

We develop on Mac machines, have our CI running in Linux and Windows and deploy to Windows (Azure).

Our deployment relies on the package-lock.json file and the appdynamics dependencies are causing us a big headache.

Is there any way that you could do something similar to the Cylon project who have multi-platform precompiled dependencies? They have documented their process in a fair amount of detail and a resulting package is node-opencv

Thank you!

Adam

Re: `NPM install` integrity check failure on Linux

Kyle_Furlong — Mon, 16 Jul 2018 16:55:42 GMT

Hi Adam,

Thanks for your interest in finding a solution. The package-lock file is definitely one that's been tricky since it came out, but one for which we've found multiple solutions.

I've skimmed through the Cylon.js post, and it looks like they do something very similar to what we provide, except they leverage node-pre-gyp to automate some of the bookkeeping. In essence both their solution and our solution provide pre-compiled binaries for your particular system by calculating your Node.js version, OS, and architecture, and downloading a pre-built binary at install time.

As I mentioned in the previous post, a lot of our customers are moving towards thinking about the appdynamics package as a service that is provided on a particular machine, not as an integral dependency in the package.json for a particular app. So the deployment process is about installing the agent globally on a target machine with npm install -g appdynamics. This avoids any package-lock issue, as well as ensuring you have the right binaries for the system. To make this work smoothly, we simply want to add an environment variable gatekeeper so that you can ensure not to call the instrumentation when the package isn't present:

if (process.env.APPD_ENABLED === 'true') {
  require('appdynamics').profile({
    ...
  });
}

What are your thoughts? We can explore other solutions in case this one is out of reach for any reason.

Regards,

Kyle

Re: `NPM install` integrity check failure on Linux

CommunityUser — Mon, 16 Jul 2018 17:02:25 GMT

Hi Kyle,

Thanks for getting back so quickly.

I hadn’t thought of putting the appdynamics require behind an environment variable switch — that makes a lot more sense of making it a per-machine installation.

One issue that this leaves outstanding for me is the issue of keeping the appdynamics client versions in sync with the central server. If we upgrade our appdynamics central server, at what point will we need to update the appdynamics clients?

Thank again,

Adam

Re: `NPM install` integrity check failure on Linux

Kyle_Furlong — Mon, 16 Jul 2018 17:09:43 GMT

Well, so our central server (what we call the controller), typically offers backwards compatibility for agents. Meaning a 4.4.x controller will support any 4.4.x and previous agents. Also! With 4.5 we're relaxing the forward compatibility restriction. 4.5+ agents will now work with any 4.4.x controllers. I would expect this going foward as well.

Long story short, if you're already on a 4.4 controller, you can and should just use the latest Node.js package.

Re: `NPM install` integrity check failure on Linux

CommunityUser — Mon, 16 Jul 2018 17:11:29 GMT

Excellent news. Thank you very much!

Re: `NPM install` integrity check failure on Linux

Kyle_Furlong — Mon, 16 Jul 2018 17:12:27 GMT

No problem! Don't hesitate to reach out again in case you need anything.

Re: `NPM install` integrity check failure on Linux

CommunityUser — Wed, 18 Jul 2018 10:52:55 GMT

Hi again Kyle,

Unfortunately, I’m still having trouble integrating the appdynamics agent into our Azure (Windows) NodeJS app service.

You proposed installing the module globally, but doing this means that the regular node require(…) does not work (see https://stackoverflow.com/a/15646750).

Unfortunately the proposed solution (using NODE_PATH to explicitly tell node of the additional global directory to search) does not work on Azure app services — the environment variable is detected but appears to have no effect.

I have even tried explicitly specifying the entire path but this failed as well — in both cases node says it cannot find the module.

My current guess is that Azure is preventing the app from loading files from outside of its wwwroot directory. Unfortunately this is where all the deployed code lives and there is already a node_modules directory there constructed from the package-json.lock using “npm install”.

So for the moment, I am unable to integrate appdynamics in our Azure app service.

I would really like to see a solution that fits the package-lock.json installation method — perhaps you could look at something similar to the way node-sass has a postinstall script that downloads the dependencies?

Thank you for your help!

Adam

Re: `NPM install` integrity check failure on Linux

Kyle_Furlong — Thu, 19 Jul 2018 16:15:18 GMT

Hi Adam,

Yikes, Microsoft does like to make our lives difficult... I definitely feel you that having AppD in your normal deployment workflow would be easiest, but unfortunately a re-architecture of our install process is probably not in the cards. It is definitely something we periodically review though—it's possible that as more and more people have end-to-end package-lock deployment workflows we'll opt to take the work on.

But for us in the mean time it means we need to keep hunting for a better process. If global install on the Azure instances isn't possible, and we have to install into the wwwroot, I wonder, could we use a post-deploy script to install the agent? That way you could have a script that basically changes to the wwwroot directory and runs npm install appdynamics@4.5.0. This same script could set the environment variable. (https://github.com/projectkudu/kudu/wiki/Post-Deployment-Action-Hooks)

Another option would be to do this one-off install at build/deploy time in your CI/CD pipeline somewhere, but depending on your environment that may take more effort. In any case, happy to keep working with you to find something that fits your needs!

Regards,

Kyle

Re: `NPM install` integrity check failure on Linux

Peter_Holditch — Tue, 17 Sep 2019 21:47:14 GMT

Just to update this thread, since customers have found it and asked for a resolution, since version 4.5.14 of the node agent, the npm install logic for the AppDynamics agent has changed to bring down the platform dependent components in a post-install script.

This means that package lock should no longer prevent Node apps built on one platform being transported to and executed on a different platform (and hence should also remedy this issue with deployment on the Azure platform)

Warm regards,

Peter.