So I’ve been converting a bunch of our jobs over to CircleCI, and with most of the smaller, simplistic jobs I’ve been impressed, the speed increase has been good, despite the complexity of the build config.
However in converting our app builds over, It’s taken me days, and I keep hitting roadblocks. My conclusion after 3 days (and a broken build) is that CircleCI 2.0 isn’t actually a good tool for testing a ‘dockerised’ application, at all.
This is largely due to the restriction of having a remote docker executor. It’s a huge hinderance, and I’ll try to explain why:
Our setup looks like this:
[ api ],
[ database ],
[ client ]
[ circleci java-container with browsers ]
In the ideal world:
First we run unit tests.
Then we build an image
Then we run functional tests against the image
Then we push the image to the docker repository
As with any good build you should run tests against a built product and deploy that product at the end.
And herein lies the first problem. We’re building a docker image between unit tests and functional tests. It makes sense to have two separate jobs for that, within a workflow.
You can’t build an image and then use it in the next job without pushing it to an external registry.
You can’t push to an external registry without giving it a unique tag otherwise a later build might overwrite it between steps.
You can’t delete tags in a registry, not without a lot of brittle, fragile, api hacking.
You shouldn’t use an external registry for 'temporary build assets’
You end up rebuilding the image in each workflow stage
But that means persisting ALL the source code at the end of each stage.
Circle’s Persisting step won’t even let you do that - it encourages specific asset persistence
Running functional tests against a composed (or dockerised) application.
It’s extremely difficult and ugly to test a dockerised application.
You can’t run up your built image from a previous step as part of the main build script, as it doesn’t exist
You have to push it externally, and then circle will pull it - but that has the same issues as in Scenario 1.
You can’t run it as part of the build or part of a compose script because it can’t see the main images (like db, etc) - you can compose all your images so that they can see eachother, BUT:
In a composed environment as part of the build, you can’t access the exposed TCP ports from the build container.
Running tests in a container is a solution but you can’t mount volumes to get your test results out, so there are no test results.
So, all in all, I’ve spent 3 days and I’m very unhappy with the solution I’ve had to come up with in order to make the build work. CircleCI 1.0 is significantly simpler and better at testing against a dockerised application, albeit slightly slower. I say slightly, because CCI 2.0 is only a minute faster overall.