How can I extract the junit XML files from within a docker container in docker?

docker
#1

I have a circle.yml file that runs things like docker-compose -f ci/docker-compose.ci.yml run web pytests. It’s working dandy and everything but… Within that container it generates a /test_artifacts/pytest.xml file that I’d like to extract and use in the CircleCI error reporting.

If volume mounts worked, I could have used that in my docker-compose.ci.yml but that’s not an option because CircleCI Docker doesn’t support volume mounts. Switching to the slower machine executor isn’t attractive.

Basically, I have a file inside that docker container I’d now like to get out. Any hot tips on how to make that work? Also, it would be nice to be able to get to that file in case the tests fail. I.e. the above docker-compose command exits !0.

#2

I should think you can do this, yep.

In the run step following your docker-compose, are your containers still running? I would suggest you make them resilient so that they are, and then you can use docker cp (from container to host) regardless of whether your tests fail or not.

Edit

Hmm, I wonder: you may not even need the containers still to be running. I seem to recall that Docker Compose will leave your containers in their old state even if all of your containers have stopped. I wonder if you can still use docker cp on them?

#3

Actually, that’s a very good start. I was able to…:

▶ docker cp $(docker-compose -f ci/docker-compose.ci.yml ps -q web):/var/lib/apt/lists/deb.debian.org_debian_dists_jessie_Release .
▶ ls -l deb.debian.org_debian_dists_jessie_Release
-rw-r--r--  1 peterbe  staff  148222 Jun 23 06:31 deb.debian.org_debian_dists_jessie_Release

That’s after the web container has been stopped. There’s hope. But I wonder, is that not just a file that is part of the image, not the container.

The next challenge is to make the file /test_artifacts/pytest.xml not disappear after the container is stopped.

#4

It should be from the container, not the image. From docker cp help:

Copy files/folders between a container and the local filesystem

Try it on a file that exists only in the container and not the image?

#5

I can do this:

▶ docker-compose -f ci/docker-compose.ci.yml run web bash
$ touch /app/foo.bar
$ exit

But as soon as I do that, that file disappears. I.e.:

▶ docker cp $(docker-compose -f ci/docker-compose.ci.yml ps -q web):/app/foo.bar .
Error: No such container:path: cfa8b245c328600acd76ab84cd894e9fb93afb110518ef8ac1942d714d9ba3eb:/app/foo.bar

The file /app/foo.bar is not part of the image, it’s part of the container and all of those files are deleted when you stop the container. It doesn’t help if I start it again with docker start $(docker-compose -f ci/docker-compose.ci.yml ps -q web).

There must be some other way to “change the image” or something inside a container.

#6

Hmm, alrighty…

Have a look at this answer, and the answer linked. Does it help if you start your exited container and then do the cp in another terminal? If you use start, can you see your touched file?

As an alternative, you could look at keeping your containers running, as I mentioned earlier.

Finally, volume mounts do seem to work in certain circumstances. I think they don’t work on running containers (because the CircleCI infra distributes them across physical machines, where volumes won’t work) but they work on data-only containers (or at least they do for me, at least in r/o mode).

#7

Starting it does not solve the problem. The files are gone. It makes sense.

That’s interesting! What I must be able to do is write to the mounted volume on the host so that it persists after the container is stopped.

I did try once with:

web:
  build:
    context: ..
    dockerfile: Dockerfile
  image: normandy:web
  volumes:
     - $PWD:/app

…which I know does not work.

#8

To resolve this, we first switched to machine instead of docker. That made it possible to volume mounts.

Then, we discovered a better way. We switch back to using docker but use a volume on each container called test-artifacts that each docker container could use. That means that files written in one docker container exists in other docker containers. Then, as a first step in the circle.yml we run:

docker-compose -f ci/docker-compose.yml run --user root artifact-collector

And in the last step we use docker cp to extract files out of that container.

See https://github.com/mozilla/normandy/blob/5eb2a4447ef3b6cf1b92c894d168b60702867eb4/.circleci/config.yml#L56-L59 and https://github.com/mozilla/normandy/blob/5eb2a4447ef3b6cf1b92c894d168b60702867eb4/ci/docker-compose.yml#L25-L32