Best practice for custom/private docker containers?


#1

Our project has a docker setup configured using a set of docker-compose.yml files. I haven’t put it in DockerHub yet, and I’m not 100% sure if I should (super new to Docker).

I don’t think I want to publish it, though, which means that I either have to use the remote docker environment (and work around the lack of volume support) or I have to use the machine executor (which will be slower to start, and possibly cost more in the future). This doesn’t seem like a great user experience – either I use a VM and throw away Circle’s sweet Docker integration, or I hack up my Docker config to be Circle-friendly, which seems to defeat the point of Docker.

Am I missing something here? Is there an easier way to do what I’m trying to do? Or perhaps there’s some way that everything private and sensitive can be kept out of the docker image, allowing it to be published?


#2

#3

Thanks for the update.

Can I use an env var for image name? E.g. I want to use an image tagged with $CIRCLE_SHA1.


#4

$CIRCLE_SHA1 isn’t available until after the ‘checkout’ step.

I’m interested to know the use case here? The base image used for testing generally stays the same so you have some consistency. Using an image based on the SHA of a previous build sounds like each job would be using a different image making it hard to get reproducible results.


#5

What I do is, I build a Docker image in the first job and then run a series of tests in the following jobs using Workflow.

I was going to rely on Docker Layer Caching, but reuse of the remote Docker env it is not guaranteed, now I push the image to a private repo and pull it in the next job.

Something looks like this:

build:
  docker: docker:17-git
  steps:
    - run: |
        docker build -t myapp:${CIRCLE_SHA1} .
        docker push my app:${CIRCLE_SHA1}

test:
  docker: docker:17-git
  steps:
    - run: |
        docker pull app:${CIRCLE_SHA1}
        docker run app:${CIRCLE_SHA1} my-test-command

Using images built with CircleCI in multiple jobs is another post that is about the same topic.


#6

@baxang thanks for the explanation.

If you’re using Workflows, you might find the ‘workspaces’ meets your need here: https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs

We built workspaces for doing this kind of thing. Would that work for you?


#7

If I was to use the Workspace, do I do docker export... in a prior job and then load it in the later jobs? (I have tried this)

Yes, I can try it but I was hoping to have a simpler way to do it.


#8

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.