Reuse docker image in same workflow

Build Environment
1

As seen in Running Docker Commands - CircleCI, how would one use this just uploaded image in a followup job? Are env variables supported in that part of the yaml, e.g. would this work?

version: 2.1
jobs:
  build:
    docker:
      - image: company/app:$CIRCLE_BRANCH
1 Like
2

I’ve posted a feature request about the same thing. It does not seem currently possible to use env variables within the circleci commands of a config.yml

What is not clear is if this covers all the commands of a config.yml as env processing could be implemented in some commands and not others - there is no clear answer that I can find to that.

3

Ok, this is a huge show stopper: How can you ever have a image tag based on some content of your repository?

4

It does seem a stupid oversight. I’m only just starting to work with circleci, but I selected circleci due to the tight integration that Doppler has with it - the inability to dynamically configure the main control file was unexpected.

Doppler is a very nice tool to manage environment variables and secrets, which I’m using to populate much of my deployed environment.

Things like image tagging do work as the command to do the tagging is going to be run in a shell where variable substitution works. So the following does what you would hope/expect

  - run:
      name: "Push Docker image"
      command: |
               docker login --username $DOCKER_USER --password $DOCKER_PASSWORD
               #docker commit $DOCKER_ACCOUNT/$PROJECT_NAME:latest
               docker tag image "$DOCKER_ACCOUNT/$PROJECT_NAME:current"
               docker tag image "$DOCKER_ACCOUNT/$PROJECT_NAME:$CIRCLE_BUILD_NUM"
               #docker tag $DOCKER_ACCOUNT/$PROJECT_NAME "$DOCKER_ACCOUNT/$PROJECT_NAME:$CIRCLE_SHA1:0:8"
               
               docker push $DOCKER_ACCOUNT/$PROJECT_NAME:$CIRCLE_BUILD_NUM
               docker push $DOCKER_ACCOUNT/$PROJECT_NAME:current

What you seem unable to do is things like the following

 executors:
   exec_node:
     machine: true
     working_directory: /LOCAL/$CIRCLE_PROJECT_REPONAME/$CIRCLE_BRANCH
5

Yes, creating the image dynamically is not an issue as you said. But it is impossible to use that just tagged image anywhere. CircleCI is really lacking many basic features :frowning:

6

You can use a setup workflow to dynamically generate a config on the fly and utilize any sort of variables you like. Think of it as writing a config that has a full execution environment to prepare the config you actually want to run. You can read more in the docs here, or view this summary with some examples.

7

Fern, thanks for the links, they may allow workarounds for certain aspects of the current circleci.yml limitations, but not the example I have been using of

 executors:
   exec_node:
     machine: true
     working_directory: /LOCAL/$CIRCLE_PROJECT_REPONAME/$CIRCLE_BRANCH

Such a step is required to take place to set up the executor environment that can then run Dynamic Configs.

My focus has been on the deployment of Self-hosted runners as I have a legacy 1.88GB docker image to rebuild in a CI pipeline and the future pricing changes mean it would be costly to use the defined CircleCI runners as the daily egress traffic is rather high.

The issue is that CircleCI runners have very good isolation as standard, with system-level caches and clean up. Self-hosted runners do not so depend on unique working directories - something that is hard to define if there is no env processing for the primary circleci.yml - or at least the working_directory command.

8

It would be great to be able to run subsequent build jobs on a previously build docker image. is that still not possible? I also tried the {{ .BRANCH }} syntax, but it doesn’t seem to get interperpreted. What a bummer :confused: