Run containers independently with parallelism?

By looking at the “build timings” tab after a build completes, it’s apparent that each container runs a step to completion, then all the containers move on to the next step only after they’ve ALL finished the first step.

This potentially wastes a lot of time, as it means the build time is the sum of the slowest steps across all containers.

It would be faster overall if each container moved on to the next step as soon as it was ready.
Then the total build time would be the max of the total times for each container, which could be a lot less.

It’s kind of like this:

>>> container1 = (10, 2)  # slow on step 1, fast on step 2
>>> container2 = (3, 10)   # fast on step 1, slow on step 2
>>> container_steps = [container1, container2]
>>> sum((max(steps) for steps in container_steps)) # this is what CCI does now
20
>>> max(map(sum, container_steps))  # this would be better
13

I don’t see any way to do this currently, am I missing something?

1 Like