Taking advantage of npm ci

Build Environment
1

npm@5.7.1 added support for npm ci, which supposedly decreases install times significantly.

However, when I tried to take advantage of it, it proved impossible. Indeed:

  • npm ci won’t work on default Circle images, as they don’t ship with that latest npm version.
  • sudo npm install npm will install, but all subsequent calls to npm will fail with Error: Cannot find module 'isarray'.
  • npm install npm; node_modules/.bin/npm ci won’t work because npm ci removes node_modules before proceeding.

I am thus back to using npm install, and will wait until Circle updates the images to the latest NPM version.

I am writing this here to:

  1. Save others’ time.
  2. In hope that someone has found a solution I haven’t tried.
  3. To draw Circle’s team attention to that topic, as it would save them resources :slight_smile:
2

You can use your own Docker image, make it build on top of the official CircleCI one and update the npm version there until the official image is updated

3

Thanks for writing this. This is helpful for those searching about this.

FYI, we don’t install npm specifically in our images, it comes from upstream. So really, we’re all waiting on when the maintainers of the Docker Library Node image, and in turn NodeJS version 8 itself will get that newer version of NPM. https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V8.md#8.10.0

4

Note about my research on npm ci and its speed:
It should be significantly faster than npm install however

npm ci always deletes and re-fetches all dependencies as specified in package-lock.json . This should in theory prevent any possible corruption of the already installed dependencies.

which is not compatible with CircleCI caches.
Caching node_modules between builds and running npm install should be way faster than npm ci.

Source:
https://medium.com/@tomastrajan/how-to-speed-up-continuous-integration-build-with-new-npm-ci-and-package-lock-json-7647f91751a

5

Note you can cache the npm cache!
It’s is stored in /root/.npm. We are successfully using npm ci with relatively quick install times using the following save_cache

- save_cache:
      key: v2-web-npm-deps-{{ .Branch }}-{{ checksum "/root/project/web/package-lock.json" }}
      paths:
        - /root/.npm
6 
# ...
    docker:
      - image: circleci/node:latest
    steps:
# ...
      - run:
          name: updating npm...
          command: npmv=$(echo $(npm -v) | head -c 1); if [ "$npmv" -lt "6" ]; then sudo npm i -g npm; else echo "Node.js Docker Team finally decided to include npm v6+ in latest image; you can remove this script now"; fi
      - run:
          name: installing dev dependencies...
          command: npm ci
# ...

I wrote this script to update npm only if it’s < 6.0.0 so I can use npm ci, and skip the update if the docker image already includes npm v6.0.0+. Hopefully, I’ll see the note once npm is updated in the docker image and I can remove the script.

I’m a beginner at circleci and yml, but I think this’ll work for the time being.

7

I spent some time figuring the npm command not found error and it turns out it was the sudo the cause so no need to use sudo:

command: npmv=$(echo $(npm -v) | head -c 1); if [ "$npmv" -lt "6" ]; then npm i -g npm; else echo "Node.js Docker Team finally decided to include npm v6+ in latest image; you can remove this script now"; fi
8

npm install -g npm is fast enough for npm install or npm ci . So it may be enough for the following code.

version: 2
jobs:
  prepare:
    working_directory: ~/foo
    docker:
    - image: circleci/node:~~~~~~~~~~
    environment:
      NPM_VERSION: "5.7.0"
    steps:
    - checkout
    - restore_cache:
        name: Restore node modules cache
        key: node_modules-{{ checksum "package-lock.json" }}
    - run:
        name: Install dependencies
        command: |
          if [ ! -d node_modules ]; then
            sudo npm install -g npm@$NPM_VERSION
            npm ci
          fi
    - save_cache:
        name: Save node modules cache
        key: node_modules-{{ checksum "package-lock.json" }}
        paths:
          - node_modules

  build:
    working_directory: ~/foo
    docker:
    - image: circleci/node:~~~~~~~~~~
    steps:
    - checkout
    - restore_cache:
        name: Restore node modules cache
        key: node_modules-{{ checksum "package-lock.json" }}
    - run:
        name: build
        command: npm run build

workflows:
  version: 2
  deploy:
    jobs:
    - prepare
    - build:
        requires:
        - prepare