# Running additional containers for Orb job **URL:** \ **Category:** Orbs\ **Created:** [June 22, 2019, 3:45pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995 "2019-06-22T15:45:55Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![tsloughter](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/tsloughter/32/15142_2.png) [@tsloughter](https://discuss.circleci.com/u/tsloughter)\ **Post date:** [June 22, 2019, 3:45pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/1 "2019-06-22T15:45:55Z")
Using my rebar3 orb [https://circleci.com/orbs/registry/orb/tsloughter/rebar3](https://circleci.com/orbs/registry/orb/tsloughter/rebar3) is less helpful when a database or any other external service that is run in a container is needed by the tests that are run. Instead of being able to use the job from the orb it requires redefining the job with the database added to images: ```auto jobs: ct: docker: - image: circleci/erlang:22 entrypoint: ["/bin/sh"] - image: circleci/postgres:11-alpine-ram environment: POSTGRES_USER: test POSTGRES_DB: test steps: - checkout - rebar3/with_deps_cache: steps: - rebar3/ct - rebar3/on_fail_store_crashdump: cmd: ct - store_test_results: path: ~/project/_build/test/logs/ - store_artifacts: path: ~/project/_build/test/logs destination: common_test # for cover - persist_to_workspace: root: ~/project/ paths: - _build/test/ ``` At first I thought using `pre-steps` would be an option. But since the docker containers started directly in a step with the remote docker are not accessible from the host that the tests are run in this is not an option for starting the database before the orb job runs tests. It would be useful if a job could be extended with additional images to run from in the workflow. ---
**Author:** ![rose](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rose/32/7816_2.png) [@rose](https://discuss.circleci.com/u/rose)\ **Post date:** [June 24, 2019, 2:17pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/2 "2019-06-24T14:17:03Z")
@tsloughter I believe you can use mustache templating syntax here… You’ll have to create an additional parameter for a possible database image, and give it an empty string as a default. The mustache syntax will act as a conditional operator and only include the requisite config.yml syntax for a second Docker image if the parameter value is non-empty. ```auto docker: - image: first-image-always-loaded <<#parameters.second-image>>- image: <><> ``` Give that a shot and see if it works for you. ---
**Author:** ![tsloughter](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/tsloughter/32/15142_2.png) [@tsloughter](https://discuss.circleci.com/u/tsloughter)\ **Post date:** [June 24, 2019, 2:41pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/3 "2019-06-24T14:41:37Z")
Right, in the orb? I can certainly extend the orb to make passing additional images a possibility. I didn’t know about the mustache templating, thanks for that, this should work fine, assuming it supports iterating over a list to have more than just 1 image and can be arbitrary yaml since the containers will need configuration. But it still means every orb has to change to include support for inserting the images through templates. It would be useful if circle supported a way to just define images you want running during a job that don’t require modifying the job itself. ---
**Author:** ![rose](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rose/32/7816_2.png) [@rose](https://discuss.circleci.com/u/rose)\ **Post date:** [June 24, 2019, 3:17pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/4 "2019-06-24T15:17:17Z")
No, iterating over a list isn’t possible, this would just allow you to have one additional image. To have more, you’d have to repeat the same logic: ```auto docker: - image: first-image-always-loaded <<#parameters.second-image>>- image: <><> <<#parameters.third-image>>- image: <><> ``` Etc. > [@tsloughter](#): > > It would be useful if circle supported a way to just define images you want running during a job that don’t require modifying the job itself. That is what executors do: [https://circleci.com/docs/2.0/reusing-config/#executor](https://circleci.com/docs/2.0/reusing-config/#executor) An executor can include any number of Docker images. ---
**Author:** ![tsloughter](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/tsloughter/32/15142_2.png) [@tsloughter](https://discuss.circleci.com/u/tsloughter)\ **Post date:** [June 24, 2019, 3:41pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/5 "2019-06-24T15:41:03Z")
The job in the orb has an executor that is defined in the orb as well. Doesn’t it end up in the same place of having to add a parameter for each individual additional image to the executor if I wanted that to be able to be extended? Hm, another option might be better than my current solution but still require a little redefining. I could make the executor a parameter (I forgot if I can just override the executor for a job or have to setup a parameter in each job for this) and the user of the orb could define their own executor and pass its name to the job as a parameter. Assuming an executor defined in the yaml using the orb can be referenced by a name passed as a parameter to an orb. It still requires the user duplicating the content of the orb executor in their own executor so it works as expected but is definitely better than what I’ve been doing so far :). So still think being able to extend would be nice but I’ll be giving this a try sometime soon and see if it works. ---
**Author:** ![rose](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rose/32/7816_2.png) [@rose](https://discuss.circleci.com/u/rose)\ **Post date:** [June 24, 2019, 3:52pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/6 "2019-06-24T15:52:13Z")
> [@tsloughter](#): > > Doesn’t it end up in the same place of having to add a parameter for each individual additional image to the executor if I wanted that to be able to be extended? Yup, no way around that. > [@tsloughter](#): > > Hm, another option might be better than my current solution but still require a little redefining. I could make the executor a parameter (I forgot if I can just override the executor for a job or have to setup a parameter in each job for this) and the user of the orb could define their own executor and pass its name to the job as a parameter. Exactly. ---
**Author:** ![tsloughter](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/tsloughter/32/15142_2.png) [@tsloughter](https://discuss.circleci.com/u/tsloughter)\ **Post date:** [June 29, 2019, 3:12pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/7 "2019-06-29T15:12:15Z")
Thanks, this worked fine. Every job in my orb now takes an executor as a parameter [https://circleci.com/orbs/registry/orb/tsloughter/rebar3](https://circleci.com/orbs/registry/orb/tsloughter/rebar3) ---
**Author:** ![bcardiff](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/bcardiff/32/17701_2.png) [@bcardiff](https://discuss.circleci.com/u/bcardiff)\ **Post date:** [January 10, 2020, 5:35pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/8 "2020-01-10T17:35:11Z")
It would be great to have a native way to extend an executor from additional docker images. And maybe a native way to extend a job doing before/after steps. These two features could allow more composability of orbs. It was great to find this topic. Thanks! ---
**Author:** ![rose](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rose/32/7816_2.png) [@rose](https://discuss.circleci.com/u/rose)\ **Post date:** [January 10, 2020, 6:06pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/9 "2020-01-10T18:06:18Z")
> [@bcardiff](#): > > It would be great to have a native way to extend an executor from additional docker images. Hi @bcardiff—you can use [mustache templating](https://mustache.github.io/mustache.5.html) to optionally extend an executor with an additional Docker image only if a certain parameter value is set, e.g.: ```yaml parameters: extra-image: type: string default: "" description: > If set, additional Docker image (including tag) to use in this executor. docker: - image: primary/image:tag<<#parameters.extra-image>> - image: <><> ``` That said, because there is no list-type parameter, and no iteration, you will have to pick a set number of optional images and image parameters to use. > [@bcardiff](#): > > And maybe a native way to extend a job doing before/after steps. Does this not suffice? [https://circleci.com/docs/2.0/reusing-config/#using-pre-and-post-steps](https://circleci.com/docs/2.0/reusing-config/#using-pre-and-post-steps) ---
**Author:** ![bcardiff](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/bcardiff/32/17701_2.png) [@bcardiff](https://discuss.circleci.com/u/bcardiff)\ **Post date:** [January 10, 2020, 6:16pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/10 "2020-01-10T18:16:24Z")
How did I missed pre / post - steps? 🙈 Yes! those suffice. Thanks! I understand the workaround for the extra-image and is useful. I wanted to vouch for a native support that would not require every author of orbs to decide/design how to pass the extra images. With apps consuming many services like db, redis, elasticsearch, etc it is easy to exceed the 1 or 2 extra images an author might provide. ---
**Author:** ![tsloughter](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/tsloughter/32/15142_2.png) [@tsloughter](https://discuss.circleci.com/u/tsloughter)\ **Post date:** [January 10, 2020, 6:32pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/11 "2020-01-10T18:32:12Z")
Agreed on still desiring a built-in way to add additional images to have running during a job. ---
**Author:** ![rose](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rose/32/7816_2.png) [@rose](https://discuss.circleci.com/u/rose)\ **Post date:** [January 10, 2020, 7:05pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/12 "2020-01-10T19:05:46Z")
Also worth pointing this out: [https://circleci.com/docs/2.0/reusing-config/#overriding-keys-when-invoking-an-executor](https://circleci.com/docs/2.0/reusing-config/#overriding-keys-when-invoking-an-executor) > When invoking an executor in a `job` any keys in the job itself will override those of the executor invoked. For example, if your job declares a `docker` stanza, it will be used, in its entirety, instead of the one in your executor. ---
**Author:** ![bcardiff](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/bcardiff/32/17701_2.png) [@bcardiff](https://discuss.circleci.com/u/bcardiff)\ **Post date:** [January 13, 2020, 1:05pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/13 "2020-01-13T13:05:54Z")
After doing some testing I have feedback for both stories: extending an executor and extending a job. The workaround for **extending an executor** using handlebars has some limitations. You are not able to pass a whole executor structure. Only a name. So it’s impossible to customize the environment of each image. IIRC there was some environment settings that are shared with all images, but even if that is possible the end result is not great. Some images could need different values for the same parameter. I think I would like to have something like the following. If `extend` exist, the `parameters` and `docker` keys could combined with clear semantics. ```yaml # disclaimer: design idea. not real config. executors: custom: extend: some_orb/base_executor docker: - environment: # environment of main image (or all images) not sure DATABASE_URL: '...' - image: mysql:5.7 environment: MYSQL_DATABASE: '...' MYSQL_ALLOW_EMPTY_PASSWORD: 'yes' ``` This will not allow to combine two executors into one. But I am happy with the scope. * * * The proposed features for pre/post steps for **extending jobs** have also some limitations. The pre/post steps are available when _invoking_ a job in a workflow, but not when _definining_ a job. When defining a job you can use parameters to allow reusability. I would like to be able to do: ```yaml # disclaimer: design idea. not real config. jobs: awesome_foo: extend: some_orb/foo pre-steps: - run: command: echo "make it awesome!" workflows: version: 2 test: jobs: - awesome_foo: name: awesome_foo_42 param_from_original_orb_foo: 42 - awesome_foo: name: awesome_foo_42 param_from_original_orb_foo: 15 ``` Without using yaml anchors (as one need to do before orbs), I think it’s impossible to define a pre/post step once and reuse it many times as in the example above. The workaround is expanding the original job steps `some_orb/foo` (and params), defeating the purpose of extensibility. * * * Thanks @rose for all the workarounds and pointing to possible directions. I wanted to sum up somewhere my last experience in case it helps some future reader. ---
**Author:** ![rose](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rose/32/7816_2.png) [@rose](https://discuss.circleci.com/u/rose)\ **Post date:** [January 13, 2020, 9:16pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/14 "2020-01-13T21:16:11Z")
> [@bcardiff](#): > > The workaround for **extending an executor** using handlebars has some limitations. > > You are not able to pass a whole executor structure. Only a name. So it’s impossible to customize the environment of each image. IIRC there was some environment settings that are shared with all images, but even if that is possible the end result is not great. Some images could need different values for the same parameter. @bcardiff there shouldn’t be any reason you can’t pass `environment` and its additional keys/values, along with your image name/tag, using the method I mentioned previously. Orbs/YAML accept multi-line parameter values. That said, [you may have to play around with the right multiline YAML syntax](https://yaml-multiline.info) to get it to interpolate correctly. > [@bcardiff](#): > > When defining a job you can use parameters to allow reusability. I would like to be able to do: > > ```auto > # disclaimer: design idea. not real config. > > jobs: > awesome_foo: > extend: some_orb/foo > pre-steps: > - run: > command: echo "make it awesome!" > > ``` How is this different than simply running `some_orb/foo` with your desired pre-steps? ---
**Author:** ![bcardiff](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/bcardiff/32/17701_2.png) [@bcardiff](https://discuss.circleci.com/u/bcardiff)\ **Post date:** [January 14, 2020, 5:53pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/15 "2020-01-14T17:53:54Z")
> [@rose](#): > > there shouldn’t be any reason you can’t pass `environment` and its additional keys/values, along with your image name/tag, using the method I mentioned previously So if the environment variable is passed as String and using the `VAR=value` syntax it seems I can do something like the following. ```auto workflows: version: 2 test: jobs: - my_orb/test: executor: name: my_orb/executor environment: > DATABASE_URL=mysql://root@localhost/sample_app extra-image: mysql:5.7 extra-image-environment: > MYSQL_DATABASE='sample_app' MYSQL_ALLOW_EMPTY_PASSWORD='yes' ``` But it is creating a ad-hoc syntax hard to follow. In my current case the base executor is only defining the docker image so it is not big deal to duplicate that. Since the docker stanza is dropped if another image is mentioned when using the orb executor, then there is no gain to use it. Regarding the extending jobs story, the difference is that the pre/post step can only be used inside the workflows jobs. If I want to use the orb job + a pre-step multiple times with different orb job parameters I would prefer to define a custom inline job extending the orb job with the pre-step and invoke multiple time the custom inline job. That can’t be done and I am forced to include the pre-step in even invocation of the orb job. The orb job is bigger than the executor so it is not an option to copy its content to the inline job. So I am failing to take some advantage of the orbs when facing the need to extending the need with additional services. ---
**Author:** ![rose](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/rose/32/7816_2.png) [@rose](https://discuss.circleci.com/u/rose)\ **Post date:** [January 14, 2020, 6:15pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/16 "2020-01-14T18:15:37Z")
> [@bcardiff](#): > > I would prefer to define a custom inline job extending the orb job with the pre-step and invoke multiple time the custom inline job. That can’t be done and I am forced to include the pre-step in even invocation of the orb job. Most orb jobs are made up of orb commands, so it shouldn’t be to hard to define your custom in-line job using whatever commands make up the orb job that you are looking to extend. ---
**Author:** ![bcardiff](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/bcardiff/32/17701_2.png) [@bcardiff](https://discuss.circleci.com/u/bcardiff)\ **Post date:** [January 14, 2020, 6:44pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/17 "2020-01-14T18:44:12Z")
Yes I agree. We’ve exposed many commands, an executor, and a job that use the above with the configurable executor as proposed. That seems the best it can get without complicating things and is great, really. The Job is a bit more complex than using commands since we aim for nice cache and test reporting out of the box, hence the value of extending it without duplicating its definition upon usage in a non simple scenario. Thanks again for all the attention @rose! ---
**Author:** ![FredrikFolkesson](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/fredrikfolkesson/32/26774_2.png) [@FredrikFolkesson](https://discuss.circleci.com/u/FredrikFolkesson)\ **Post date:** [February 19, 2021, 4:20pm UTC](https://discuss.circleci.com/t/running-additional-containers-for-orb-job/30995/18 "2021-02-19T16:20:40Z")
Hi! I don’t know if I should create a new thread or necro this one. But I have a question on how to set environment variables for extra images. What I am trying to do is this: ```auto docker: - image: <> <<#parameters.postgres-image>>- image: <> environment: POSTGRES_PASSWORD: password POSTGRES_DB: <> <> ``` Where I have the parameters: jdk image: for the main image to use service-name: the name of the service postgres-image: for the postgres image to use. What I want is that if you supply a postgres image it should be started and the environment variables should be set. But I am getting the error `syntax error: expected , but found '?'` for the yaml linter and the pack command also throws an error `Error: An unexpected error occurred: yaml: line 4: did not find expected '-' indicator`