Caching between different workflows

I’ve recently started experimenting with breaking down our massive single workflow into a couple. The major reason for the changes is to get the ability to establish two “workspaces”: one with production dependencies and another with development/test dependencies.

Before:

  1. Install production, development, and test dependencies
  2. Run all unit tests
  3. Run all end-to-end tests
  4. Deploy related jobs

After:

  1. Production workflow
    1. Install dependencies
    2. Run end-to-end tests
    3. Deploy related jobs
  2. Development workflow
    1. Install dependencies
    2. Run all tests (6+ jobs)

I have a couple of questions:

  1. Can I use a cache from 1 workflow in another?

    I want to leverage the installed dependencies from production workflow in the development workflow (because the production set is a subset of the development set). Ideally, I would also make the install jobs dependent on each other.

  2. Can I ensure a job from 1 workflow waits on another job from another workflow?

1 Like

Hi @juanca! That’s great to hear you are making use of multiple workflows, hopefully my suggestions below can benefit your use case. To answer your questions:

  1. Can I use a cache from 1 workflow in another
    Yes! You will need to utilize the save_cache and restore_cache steps in order to save and restore the cache. Please note caches are immutable so you will need to increment the cache key in order to create a new cache. Additionally, caches expire in 15 days. You can read more about cache here: Caching, save_cache step, restore_cache step

  2. Can I ensure a job from 1 workflow waits on another job from another workflow?
    For this you can use the v2 API and Pipeline Parameters to trigger a workflow to begin once the when condition has been met. This ensures your development workflow will only run once production has completed.

version: 2.1

parameters:
  run_development_workflow:
    type: boolean
    default: false

workflows:
  version: 2
  development_workflow:
    when: << pipeline.parameters.run_development_workflow >>
    jobs:
      - mytestjob

jobs:
...

Example API request and parameters passed, where the run_development_parameter parameter is passed as true and therefore will run the development_workflow based on the when condition being met.

curl -u ${CIRCLECI_TOKEN}: -X POST --header "Content-Type: application/json" -d '{
  "parameters": {
    "run_development_workflow": "true"
  }
}' https://circleci.com/api/v2/project/:project_slug/pipeline

You can read more about triggering workflows with the v2 API and Pipeline Parameters here: Passing parameters when triggering pipelines via the API and our v2 API

1 Like