# Dynamically set parameter values of matrix

**URL:** <https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172>\
**Category:** Build Environment\
**Tags:** 2.0, circle-yml\
**Created:** [February 22, 2023, 12:01pm UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172 "2023-02-22T12:01:38Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ipap](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ipap/32/32413_2.png) [@ipap](https://discuss.circleci.com/u/ipap)\
**Post date:** [February 22, 2023, 12:01pm UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172/1 "2023-02-22T12:01:38Z")

</div>

Hi,

My use case is this:  
I run a script to generate an array of versions. I want these values to be used as parameters in a matrix so that I can run my `test` job for each one. I do not know the values before running the pipeline, the script generates them dynamically and they can change anytime as they are an external dependecy.

Let’s say ‘build-matrix’ is the job that runs the script and generates the version and ‘test’ is the job I want to run as a parameterised matrix.

I haven’t figured out how this is possible, as I assume the matrix parameters are expected to be set when the container starts, and the script in my other job (build-matrix) hasn’t run yet.

Is there any way to do this? An example mock snippet:

```auto
....
jobs:
  build-matrix:
    steps:
      - checkout
            - run:
          name: get versions
          command: | 
            TARGET_VERSIONS=$(script-to-generate-versions.sh)
            echo "export TARGET_VERSIONS='$TARGET_VERSIONS'" >> "$BASH_ENV"
  test:
    parameters:
      my_version:
        type: string
    - steps:
       ....

workflows:
  test:
    jobs:
      - build-matrix
      - test:
          name: test-<< matrix.my_version >>
          matrix:
            parameters:
              engine_version: $TARGET_VERSIONS
          requires:
            - build-matrix

....

```

We have achieved the same behaviour in Github workflows by setting the output of the first job in “outputs” and then using that in the strategy matrix.

---

<div class="post-metadata">

**Author:** ![jerdog](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/jerdog/32/32130_2.png) [@jerdog](https://discuss.circleci.com/u/jerdog)\
**Post date:** [March 8, 2023, 5:04pm UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172/2 "2023-03-08T17:04:25Z")

</div>

Sorry for the delay in a response @ipap

If I’m understanding what you’re wanting to do, you can do this with Matrix jobs, making sure to call the parameters and instantiate them as needed. Here are some things to use as examples and should get you down the line here:

> **[Matrix jobs - simple configuration for complicated software](https://circleci.com/blog/circleci-matrix-jobs/)**
>
> Learn how to use CircleCI matrix jobs to write simple configuration for complicated software.

> **[Managing reusable pipeline configuration with object parameters](https://circleci.com/blog/parameters-in-pipeline-config/)**
>
> Implementing object parameters to reuse code in pipeline configuration files.

> **[Selecting a workflow to run using pipeline parameters - CircleCI](https://circleci.com/docs/selecting-a-workflow-to-run-using-pipeline-parameters/)**
>
> A how-to guide for selecting a workflow to run using pipeline parameters.

> **[Building CI/CD pipelines using dynamic config](https://circleci.com/blog/building-cicd-pipelines-using-dynamic-config/)**
>
> Learn how to build CI/CD pipelines using CircleCI dynamic configuration.

---

<div class="post-metadata">

**Author:** ![ipap](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ipap/32/32413_2.png) [@ipap](https://discuss.circleci.com/u/ipap)\
**Post date:** [March 8, 2023, 5:25pm UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172/3 "2023-03-08T17:25:11Z")

</div>

Hi @jerdog , I managed to do this by using a dynamic config. I first run a script to generate the values I need and then pass them as env variables to a “generate-config” bash script. Then I expand those values in the matrix.

The tricky part is that the continuation config accepts just a json object, whereas I wanted to pass something like this: [2.3 , 5.6, 5.7, 8.0] .  
A helpful video was this:

[![](https://global.discourse-cdn.com/circleci/original/3X/4/c/4ceed0106387e5e8c987fda047fe258cad76a294.jpeg "Intro to Dynamic Configuration") ](https://www.youtube.com/watch?v=ytnOFfrV_Xs)

where they left the parameters as an empty object: `parameters: {}` and they passed the variable needed as a en env var to the bash script.

---

<div class="post-metadata">

**Author:** ![jerdog](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/jerdog/32/32130_2.png) [@jerdog](https://discuss.circleci.com/u/jerdog)\
**Post date:** [March 8, 2023, 6:51pm UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172/4 "2023-03-08T18:51:43Z")

</div>

Yep that makes a lot of sense! Do you think you could share your config (sanitised from secrets, etc.) to give others an example?

---

<div class="post-metadata">

**Author:** ![ipap](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ipap/32/32413_2.png) [@ipap](https://discuss.circleci.com/u/ipap)\
**Post date:** [March 10, 2023, 10:47am UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172/5 "2023-03-10T10:47:49Z")

</div>

Sure. It looks like this:

I have one “base” config: config.yml, where I used the continuation orb.

1. I calculate the versions I need
2. I save those in an env var and pass it into a bash script (can be any script)
3. I use the continuation orb to specify the next config.

- As you notice, I leave parameters with the default empty { } as my target versions are in the format of "[“v1.0.0”, “v1.2.1”, “v.1.3.1”] and parameters accepts only a JSON object. It didn’t make sense for me to make it a JSON object for this and it was easier to setup. (if you know a smarter way let me know!)

```auto
# this allows us to use CircleCI's dynamic configuration feature
setup: true

orbs:
  continuation: circleci/continuation@0.3.1

workflows:
  setup:
    jobs:
      - setup

jobs:
  setup:
    <<: *defaults
    resource_class: small
    steps:
      - checkout
      - run:
          name: calculate versions
          command: |
            # mock code to calculate TARGET_VERSIONS
            echo "export TARGET_VERSIONS='$TARGET_VERSIONS'" >> "$BASH_ENV"
      - run:
          name: Generate Pipeline continue_config.yml file
          command: |
            ./scripts/ci/generate_test_config.sh ${TARGET_VERSIONS}
      - continuation/continue:
          configuration_path: .circleci/continue_config.yml
          parameters: '{}'

```

Then in the `continue_config.yml` config I have everything I need, and in the place of the matrix, I have left an empty but it doesn’t matter what you put there as it will be replaced by the script:

```auto
workflows:
  test:
    jobs:
      - test:
          matrix:
            parameters:
              # This value will be replaced by the generate_test_config.sh script in the CI
              target-version: []
jobs:
  test:
    resource_class: medium
    parameters:
      target-version:
        type: string
    steps:
      ....

```

now in the bash script itself. you can do anything, I just used `yq` to edit the one line I needed in place:

```auto
#!/usr/bin/env bash
set -o errexit -o pipefail

target_versions="$TARGET_VERSIONS"

echo "Installing yq..."
curl -s -L -o yq "https://github.com/mikefarah/yq/releases/download/v4.31.2/yq_linux_amd64"
chmod +x ./yq
mkdir -p "${HOME}/.local/bin"
mv ./yq "${HOME}/.local/bin/"

# We use yq to replace the target-version key, this is the only edit in place we need
yq e ".workflows.test.jobs[0].test.matrix.parameters.\"target-version\" = $target_versions" -i .circleci/continue_config.yml

# (optionally install aws cli tool too and validate the config you generated)
# circleci config validate .circleci/continue_config.yml 

```

---

<div class="post-metadata">

**Author:** ![ipap](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ipap/32/32413_2.png) [@ipap](https://discuss.circleci.com/u/ipap)\
**Post date:** [March 10, 2023, 10:49am UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172/6 "2023-03-10T10:49:06Z")

</div>

@jerdog let me know if something doesn’t make sense, or can be refactored! Thanks

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/circleci/original/3X/5/1/511608acda8e85906e15909b57bd0a46c1dbe291.png) [@system](https://discuss.circleci.com/u/system)\
**Post date:** [March 17, 2023, 10:49am UTC](https://discuss.circleci.com/t/dynamically-set-parameter-values-of-matrix/47172/7 "2023-03-17T10:49:53Z")

</div>

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.