Attaching workspace has an unpredicably different timing

Build Environment
1

Hi, we have a single install job and then run tests job in the same workflow v2.1
We split the tests in the test job using parallelism flag, usually running 4 boxes.
The install job does NPM install and passes the workspace to the test boxes.

Now what we have noticed was that each of the test boxes can take a widely different
time to attach the same workspace. Typical values are from 18 seconds to 1.5 minutes.
This causes our test parallelization to be suboptimal since some boxes starting
the tests much later :frowning:

An example https://app.circleci.com/pipelines/github/cypress-io/cypress-documentation/6188/workflows/64248a94-a577-4628-b577-e5f8842f1ebc/jobs/27738/parallel-runs/0?filterBy=ALL

5 parallel boxes. Workspace attachment

Downloading workspace layers
workflows/workspaces/cbe9c52f-1583-4b2a-8acd-71d959fcac7c/0/4aa55480-159c-49d9-865a-c2fc8342eec9/0/119.tar.gz - 449 MB
Applying workspace layers
4aa55480-159c-49d9-865a-c2fc8342eec9

Time it took to attach the workspace: 11s, 11s, 24s, 1m3s, 1m40s

Is there anything we can do to normalize the attachment time and make it more uniform?

The workspace is about

2

Hello,

There indeed is something you can do to potentially normalize the times better here. Skip the disk.

You can set up a ramdisk and save all of your workspace artifacts there. This should improve performance and consistency across jobs that utilize the workspace. The files stored there will skip the disk when they are persisted or loaded from the workspace. Just make sure to configure it on each job that uses the workspaces.

1 Like
3

Nice, this seems to work pretty well, as I described in Start CircleCI Machines Faster by Using RAM Disk | Better world by better software and we are switching our projects to use RAM disk now

1 Like
4

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.