Odd behavior with Continuation and Trigger Pipeline from CircleCI UI

Feedback & Bug Reports
1

I had a heck of a time figuring out what was going on and how to fix this problem. I hope it wasn’t obvious from the docs and I just missed it.

We had a pipeline and we wanted to figure out a count to do some parallelism in a job. The only way we could figure out how to make this work was with an initial pipeline that figured out a count and use Continuation to call the main pipeline with that new count as a parameter. All well and good.

However we had a couple of parameters, run_failed_testcases and failed_doctor_workflow_id, that were only used by our QA team to signal the pipeline to only re-run particular tests. These were sent by using the Trigger Pipeline from the CircleCI UI and setting the two parameters.

Initially we just copied the two parameters and their defaults from our main pipeline file to the setup and add them to the parameter JSON given to the continuation/continue.

Well, that lead us to find that we could not use the same parameter names in the setup and the main. (We did find a post indicating this after the fact.) So after updating the names in the main and the generated JSON of the setup to run_failed_testcases_from_setup and failed_doctor_workflow_id_from_setup, the pipelines would pass normally.

When we tried to trigger the pipelines to do the selective test rerun, we got the Unexpected argument(s): run_failed_testcases, failed_doctor_workflow_id?!?

The only solution we could come up with was to add a parameter definition for the original run_failed_testcases and failed_doctor_workflow_id parameters to the main workflow yaml. It was as if, the UI managed to send the parameters set in the UI to both the setup and the main yaml files.

Since our workflow files are fairly complex, here is a set of very simplified files that demonstrate the behavior. I hope this either helps someone or CircleCI can explain the behavior.

Here is a sample representing our initial workflow. Even though hello_name is in this example, we will use it in the following as the ‘new’ parameter calculated by the setup. The triggered represents one of the QA parameters.

version: 2.1

parameters:
  hello_name:
    type: string
    default: "world"
  triggered:
    type: string
    default: "no"

jobs:
  say_hello:
    docker:
    - image: cimg/base:stable
    resource_class: small
    steps:
      - run: echo "Hello << pipeline.parameters.hello_name >>!"
      - run: echo "Was I triggered? << pipeline.parameters.triggered >>!"

workflows:
  main:
    jobs:
      - say_hello

This works fine. We can run it and trigger it from the UI.

This represents our first simple attempt.

We renamed the above from config.yml to main.yml. The new config.yml is represented here:

version: 2.1

setup: true

orbs:
  continuation: circleci/continuation@1.0.0

parameters:
  hello_name:
    type: string
    default: "none"
  triggered:
    type: string
    default: "no, but from setup"

jobs:
  do_prep:
    docker:
      - image: cimg/base:stable
    resource_class: small
    steps:
      - checkout
      - run:
          name: Pass through the parameters for followup testing_app
          command: |
            echo "{\"hello_name\": \"<< pipeline.parameters.hello_name >>\"," > pipeline-parameters.json
            echo " \"triggered\":  \"<< pipeline.parameters.triggered >>\" }" >> pipeline-parameters.json
      - continuation/continue:
          configuration_path: .circleci/main.yml
          parameters: pipeline-parameters.json

workflows:
  setup:
    jobs:
      - do_prep

This was when we found out that the parameter names could not be the same in both, sigh. This was the next iteration.

config.yml (note the addition of _from_setup to the parameters passed in)

version: 2.1

setup: true

orbs:
  continuation: circleci/continuation@1.0.0

parameters:
  hello_name:
    type: string
    default: "none"
  triggered:
    type: string
    default: "no, but from setup"

jobs:
  do_prep:
    docker:
      - image: cimg/base:stable
    resource_class: small
    steps:
      - checkout
      - run:
          name: Pass through the parameters for followup testing_app
          command: |
            echo "{\"hello_name_from_setup\": \"<< pipeline.parameters.hello_name >>\"," > pipeline-parameters.json
            echo " \"triggered_from_setup\":  \"<< pipeline.parameters.triggered >>\" }" >> pipeline-parameters.json
      - continuation/continue:
          configuration_path: .circleci/main.yml
          parameters: pipeline-parameters.json

workflows:
  setup:
    jobs:
      - do_prep

This is the new main.yml (again, note the added _from_setup suffix to the parameters).

version: 2.1

parameters:
  hello_name_from_setup:
    type: string
    default: "world"
  triggered_from_setup:
    type: string
    default: "no"

jobs:
  say_hello:
    docker:
    - image: cimg/base:stable
    resource_class: small
    steps:
      - run: echo "Hello << pipeline.parameters.hello_name_from_setup >>!"
      - run: echo "Was I triggered? << pipeline.parameters.triggered_from_setup >>!"

workflows:
  main:
    jobs:
      - say_hello

This worked fine when run ‘normally’, say from a push to a monitored branch of a git repo. However, if we tried to trigger from the CircleCI UI with, for example, triggered = YES, we received: Unexpected argument(s): triggered.

The final solution was to add the parameters without the suffix back into the main.yml as:

version: 2.1

parameters:
  hello_name_from_setup:
    type: string
    default: "world"
  triggered_from_setup:
    type: string
    default: "no"
  hello_name:
    type: string
    default: "silly hello_name"
  triggered:
    type: string
    default: "silly triggered"

jobs:
  say_hello:
    docker:
    - image: cimg/base:stable
    resource_class: small
    steps:
      - run: echo "Hello << pipeline.parameters.hello_name_from_setup >>!"
      - run: echo "Was I triggered? << pipeline.parameters.triggered_from_setup >>!"

workflows:
  main:
    jobs:
      - say_hello

Now the workflow will run ‘normally’ and it will work as expected when the parameters are set from the Trigger Pipeline in the CircleCI UI.

Cheers
Kevin