Yarn cached install slow

Build Environment Caching Dependencies
, , , ,
1

Hey there! I’m currently trying to use caching to reduce the amount of time installing dependencies takes in my builds, but I’m seeing that even with caching implemented as suggested, yarn install is still taking ~1 minute, which is quite long, considering the restore_cache step is saying that it found an applicable cache. Does anyone see anything specifically wrong with my caching setup here?

Here’s my config: (these are steps in my main test build job)

- restore_cache:
    name: Restore yarn package cache
    keys:
      - yarn-packages-{{ checksum "yarn.lock" }}

- run:
    name: Install node modules
    command: yarn install --frozen-lockfile

- save_cache:
    name: Save yarn package cache
    key: yarn-packages-{{ checksum "yarn.lock" }}
    paths:
      - ~/.cache/yarn

and then yarn outputs:

#!/bin/bash -eo pipefail
yarn install --frozen-lockfile
yarn install v1.22.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@1.2.9: The platform "linux" is incompatible with this module.
info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
<various peer dependency warnings>
e.g.
warning " > babel-loader@8.0.6" has unmet peer dependency "webpack@>=2".
[4/4] Building fresh packages...
Done in 50.98s.
CircleCI received exit code 0

The cause of it taking 51 seconds eludes me! Locally, it takes 0.6s, so I thought it might be yarn’s internal registry cache, but even after clearing that with yarn cache clear, it was the same time. Am I missing something? I’d like to be able to knock a full minute off my build if possible. Let me know if whoever reads this needs more information as well, but I think this should be all of the pertinent config/logs.

2

Ran into the same issue and after a long while of debugging was able to isolate this to two issues which you may also be facing.

Through using the --verbose flag on the yarn install I realized most of the time was spent fetching packages, indicating the cache wasn’t loading correctly. The solution to this first issue was due to installing yarn after restoring the cache. We were installing a version of yarn that used a different cache directory, so moving the yarn install to before the restore_cache step helped here.

But after this, the timing did not decrease significantly. Now we realized that a majority of the install time was spent linking dependencies. This was because we were not caching the node_modules (and if you utilize yarn workspaces, the subsequent workspace node_modules link). Without caching the node_modules as well, the install still takes a lot of time on i/o operations to move the files around.

If you use yarn workspaces then you need something like:

    - save_cache:
        key: <key>
        paths:
          - node_modules
          - workspace1/node_modules
          - workspace2/node_modules

Hope this helps!

1 Like