I need to run benchmark tests as part of a CI run. These tests are performance-sensitive, and so they require separate VMs for each component (database, search engine, etc.).
From what I can tell, each job can only run on one machine. Is there no support for provisioning multiple machine resources and scheduling steps to run on them?
You should be able to make use of matrix jobs to specify a matrix of resource classes:
As an example, this would build on the medium and large resource classes with machine executor:
resource_class: << parameters.resource >>
- run: echo 1
resource: ["medium", "large"]
That only runs the test on one machine at a time, though, as far as I can see. I’m asking about starting multiple machines during a single build, and scheduling different job steps on them. Considering that CircleCI supports detached steps with
background: true, I’m surprised there’s no way to tell it where to run the steps.