Running additional containers for Orb job

Using my rebar3 orb https://circleci.com/orbs/registry/orb/tsloughter/rebar3 is less helpful when a database or any other external service that is run in a container is needed by the tests that are run.

Instead of being able to use the job from the orb it requires redefining the job with the database added to images:

jobs:
  ct:
    docker:
      - image: circleci/erlang:22
        entrypoint: ["/bin/sh"]
      - image: circleci/postgres:11-alpine-ram
        environment:
          POSTGRES_USER: test
          POSTGRES_DB: test
    steps:
      - checkout
      - rebar3/with_deps_cache:
          steps:
            - rebar3/ct
      - rebar3/on_fail_store_crashdump:
          cmd: ct
      - store_test_results:
          path: ~/project/_build/test/logs/
      - store_artifacts:
          path: ~/project/_build/test/logs
          destination: common_test
      # for cover
      - persist_to_workspace:
          root: ~/project/
          paths:
            - _build/test/

At first I thought using pre-steps would be an option. But since the docker containers started directly in a step with the remote docker are not accessible from the host that the tests are run in this is not an option for starting the database before the orb job runs tests.

It would be useful if a job could be extended with additional images to run from in the workflow.

@tsloughter I believe you can use mustache templating syntax here… You’ll have to create an additional parameter for a possible database image, and give it an empty string as a default. The mustache syntax will act as a conditional operator and only include the requisite config.yml syntax for a second Docker image if the parameter value is non-empty.

docker:
  - image: first-image-always-loaded
  <<#parameters.second-image>>- image: <<parameters.second-image>><</parameters.second-image>>

Give that a shot and see if it works for you.

Right, in the orb? I can certainly extend the orb to make passing additional images a possibility. I didn’t know about the mustache templating, thanks for that, this should work fine, assuming it supports iterating over a list to have more than just 1 image and can be arbitrary yaml since the containers will need configuration.

But it still means every orb has to change to include support for inserting the images through templates. It would be useful if circle supported a way to just define images you want running during a job that don’t require modifying the job itself.

No, iterating over a list isn’t possible, this would just allow you to have one additional image. To have more, you’d have to repeat the same logic:

docker:
  - image: first-image-always-loaded
  <<#parameters.second-image>>- image: <<parameters.second-image>><</parameters.second-image>>
  <<#parameters.third-image>>- image: <<parameters.third-image>><</parameters.third-image>>

Etc.

That is what executors do:

https://circleci.com/docs/2.0/reusing-config/#executor

An executor can include any number of Docker images.

The job in the orb has an executor that is defined in the orb as well. Doesn’t it end up in the same place of having to add a parameter for each individual additional image to the executor if I wanted that to be able to be extended?

Hm, another option might be better than my current solution but still require a little redefining. I could make the executor a parameter (I forgot if I can just override the executor for a job or have to setup a parameter in each job for this) and the user of the orb could define their own executor and pass its name to the job as a parameter. Assuming an executor defined in the yaml using the orb can be referenced by a name passed as a parameter to an orb.

It still requires the user duplicating the content of the orb executor in their own executor so it works as expected but is definitely better than what I’ve been doing so far :). So still think being able to extend would be nice but I’ll be giving this a try sometime soon and see if it works.

Yup, no way around that.

Exactly.

Thanks, this worked fine. Every job in my orb now takes an executor as a parameter https://circleci.com/orbs/registry/orb/tsloughter/rebar3