How to use attach_workspace and parallelism without attaching all the parallel containers' workspaces?


#1

In my build job, I use parallelism: 6 and I persist the workspace using persist_workspace.

In another workflow job, when I use attach_workspace, it seems to attach all 6 workspaces from the 6 parallel running containers (taking almost 50 seconds). I only need one of them, as I only need one copy (in this case, the gems).

How can I specify that I only want to persist and attach one copy?

Build:

    parallelism: 6
    steps:
      - persist_to_workspace:
              root: .
              paths:
                - vendor/bundle

Later step in the workflow:

  - attach_workspace:
          at: .

Log output from ‘Attaching Workspace’ step:

Downloading workspace layers
  workflows/workspaces/b92dbe----/0/690d---/0/106.tar.gz - 297 MB
  workflows/workspaces/b92dbe----/0/690d---/1/106.tar.gz - 297 MB
  workflows/workspaces/b92dbe----/0/690d---/2/106.tar.gz - 297 MB
  workflows/workspaces/b92dbe---/0/690d---/3/106.tar.gz - 297 MB
  workflows/workspaces/b92dbe---0/690d---/4/106.tar.gz - 297 MB
  workflows/workspaces/b92dbe---/0/690d---/5/106.tar.gz - 297 MB
Applying workspace layers
  690d----

#2

The approach I’ve used is to have a 1st build that does bundle install, then use save_cache to save the gems. After that, restore_cache in each build step.

In the workflow configuration, make the do_something (parallelism 6) job dependent on bundle_install.

You could just as easily use persist_to_workspace if you require a fresh bundle install for each build. I didn’t find any value in a fresh bundle install as long as I cached with a key based on {{ checksum "Gemfile.lock" }}


#3

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.