Caching between different workflows

[juanca]

I’ve recently started experimenting with breaking down our massive single workflow into a couple. The major reason for the changes is to get the ability to establish two “workspaces”: one with production dependencies and another with development/test dependencies.

Before:

1. Install production, development, and test dependencies
2. Run all unit tests
3. Run all end-to-end tests
4. Deploy related jobs

After:

1. Production workflow
   1. Install dependencies
   2. Run end-to-end tests
   3. Deploy related jobs
2. Development workflow
   1. Install dependencies
   2. Run all tests (6+ jobs)

I have a couple of questions:

1. Can I use a cache from 1 workflow in another?

   I want to leverage the installed dependencies from production workflow in the development workflow (because the production set is a subset of the development set). Ideally, I would also make the install jobs dependent on each other.

2. Can I ensure a job from 1 workflow waits on another job from another workflow?

[angelisa]

Hi @juanca! That’s great to hear you are making use of multiple workflows, hopefully my suggestions below can benefit your use case. To answer your questions:

1. Can I use a cache from 1 workflow in another
   Yes! You will need to utilize the save_cache and restore_cache steps in order to save and restore the cache. Please note caches are immutable so you will need to increment the cache key in order to create a new cache. Additionally, caches expire in 15 days. You can read more about cache here: Caching, save_cache step, restore_cache step

2. Can I ensure a job from 1 workflow waits on another job from another workflow?
   For this you can use the v2 API and Pipeline Parameters to trigger a workflow to begin once the when condition has been met. This ensures your development workflow will only run once production has completed.

version: 2.1

parameters:
  run_development_workflow:
    type: boolean
    default: false

workflows:
  version: 2
  development_workflow:
    when: << pipeline.parameters.run_development_workflow >>
    jobs:
      - mytestjob

jobs:
...

Example API request and parameters passed, where the run_development_parameter parameter is passed as true and therefore will run the development_workflow based on the when condition being met.

curl -u ${CIRCLECI_TOKEN}: -X POST --header "Content-Type: application/json" -d '{
  "parameters": {
    "run_development_workflow": "true"
  }
}' https://circleci.com/api/v2/project/:project_slug/pipeline

You can read more about triggering workflows with the v2 API and Pipeline Parameters here: Passing parameters when triggering pipelines via the API and our v2 API