Workflows needs a 'previous image' directive

build-image
cache
workflow
2.0
apt

#1

Workflows seem pretty nice at a first glance, but there is a feature that is lacking that makes them nearly useless for C++ development - the ability to specify that a job should run on an output image from a workflow. Since everything is in docker anyway, I don’t see why this would be too hard to implement.

For example: the fan out/fan in example: https://github.com/CircleCI-Public/circleci-demo-workflows/blob/fan-in-fan-out/.circleci/config.yml uses save_cache/load_cache to put dependencies into a starter step, but for C++ and mixed app development, we really want to just build an image in the first step (by installing packages normally) and use the said container in subsequent jobs. this way, we can install packages from any package manager (including the native one) and there is no possibility of missing some folders, as there is with save-cache.

Another way to think of this is the ‘pipeline’ metaphor, where a container flows down the pipeline, every job running more and more commands until you have a package at the end.

Let me know what you think, or if this feature is actually already implemented!


Docker images between jobs in Workflows
#2

Also, if anyone knows how to deploy the current running image to the docker hub (providing a hack to get this), that would be appreciated.

I’m kind of surprised (or maybe I’m just missing it) that there’s no way to deploy an image built in a step, because that would be a natrual deploy mechanism for a docker based build.


#3

Just my 2c, but the “reusable” directive, should be reusing remote_dockers from previous jobs in the same workflow instead of “random” remote_dockers that have been associated with the account/repo before (that’s at least what I’m seeing).
That way you’ll make sure that build containers in previous jobs are available as is in consecutive jobs consistently.


Caching Docker images in Workflows
#4

I wanted to add a bit more context for this feature request.

A while back (2 years now) I had the idea of running tests in this way, and I made a simple set of scripts around CircleCi to make it happen. By running tests in a docker image, it’s able to cache dependencies for C++ projects, imrpoving their build times significantly (over a 70% reduction in my cases), while still allowing for people to transparently update dependencies by updating an install script or similar.

While the implementation quality might be lacking, I think that the core ideas behind DoCIF are amazing, and that more people should use them, and that’s why I would like a feature like this in CircleCI 2.0, so that we can pull images from a cache and rebuild and push to docker hub auto-magically when the cache is invalidated. I also would love to port DoCIF to circleci 2.0, but right now the features that it needs aren’t available (dynamic pulling of images and pushing the running container to the docker hub).


#5