Destroy npm cache (./node_modules/) based on a hash of package.json (or, if present: npm-shrinkwrap.json)?

Build Environment
, ,
1

We’ve been trying to understand how Circle cache busting works in the context of node_modules.

Whenever we introduce a new dependency to our package.json (& npm-shrinkwrap.json), our build breaks for that branch and we must click “Rebuild without cache” on the branch build UI. We frequently have to do this again when this branch is merged into master.

This was much worse (completely unusable) when we were using bcrypt and other native C++ modules that required node-gyp to build.

I suspect we’re doing something wrong here, but am not sure what it is. Any help would be appreciated.

5 Likes
2

I actually have the exact same question as well - I’d expect a change to package.json or npm-shrinkwrap.json to bust the node_modules cache and run a re-install. Happy to hear any advice, hope this comment helps bump up this post to the top again for an answer.

1 Like
3

I would try:

dependencies:
  pre:
    - npm install -g npm
  override:
    - npm update

The pre install the latest npm
The override make sure you have the latest deps

Note: Make sure to run the first build without the cache

4

I’m having a similar problem. When a package is removed via npm uninstall and a new shrinkwrap is generated, CircleCI seems to use the cached node_modules. It seems like it should notice the package.json or shrinkwrap has changed and re-run npm install. I only discovered this because I was testing that if there was a required package missing then the build would fail.

I don’t understand the suggestion to run npm update. That’s just going to install the latest versions. That is not at all what we want to happen.

5

To me npm install should at least check if there is a newer version available of an already cached version. The current implementation does not do that and this is wrong!

6

Is this still not supported? Would love this feature.

7

We’re having exactly the same issue. It means every time we branch, we’ve got to remember to rebuilt without the cache.

Here’s our original forum posting: Npm bcrypt binary not saved to cache?

Is there any work around at all for this?

8

Getting similar error. We use greenkeeper to keep up to date NPM packages and seem to be running into a cache issue after the npm install -g command runs

tried the update suggestion but that didn’t work

npm ERR! Linux 3.13.0-106-generic
npm ERR! argv "/home/ubuntu/nvm/versions/node/v6.1.0/bin/node" "/home/ubuntu/nvm/versions/node/v6.1.0/bin/npm" "update"
npm ERR! node v6.1.0
npm ERR! npm  v3.3.12
npm ERR! path /home/ubuntu/-api-data/node_modules/firebase/node_modules/faye-websocket/node_modules/websocket-driver/node_modules/websocket-extensions
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename

npm ERR! enoent ENOENT: no such file or directory, rename '/home/ubuntu/-api-data/node_modules/firebase/node_modules/faye-websocket/node_modules/websocket-driver/node_modules/websocket-extensions' -> '/home/ubuntu/-api-data/node_modules/firebase/node_modules/websocket-extensions'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent 

npm ERR! Please include the following file with any support request:
npm ERR!     /home/ubuntu/-api-data/npm-debug.log

npm update returned exit code 254

Action failed: npm update
9

Circle should not cache node_modules, it should cache the npm cache. Is there any way to disable caching node_modules? I have to rebuild without cache almost every time with npm 5’s package-lock.json.