Feedback wanted: moving from v1.1 job triggering to v2 pipeline triggering

[ndintenfass]

With API v2 emerging into production use we are planning to ask everyone who uses the v1.1 job triggering endpoint to move over the new method of triggering in the v2 API. We will not be immediately disabling the v1.1 endpoint for triggering a job, but for a variety of reasons we would prefer to turn it off in a few months (current target is March 2020, but that is subject to feedback from those reading this).

We think the new pipeline triggering is superior for a variety of reasons, some for our internal operation and some for API developers. The benefits to you are:

1. The new triggering endpoint offers pipeline parameters, which are more explicit and expressive.
2. The new triggering allows you to run workflows
3. The new triggering works with v2.1 config, allowing use of reusable config elements, including orbs
4. The new triggering works best with our new pipelines UI

For those using build parameters in the v1.1 job triggering endpoint you will need to make some slight configuration adjustments to either explicitly inject the environment variables that represent the build parameters used in your jobs or swap out the environment variables with the pipeline variable syntax (eg: << pipeline.parameters.mything >>). I am eager to hear from anyone who finds this onerous, as there are things we might consider if it turns out this is problematic due to issues of scale (for instance, if you have lots of projects to change I’d like to hear from you to talk about how we might be able to help).

I am starting this topic as a place to talk about how to make the transition and to encourage discussion from you if you run into problems or find the new mechanism insufficient to make the switch.

I also hope to solicit input on whether anyone reading this feels like they need more time than March 2020 to stop using the job triggering in v1.1 and switch to pipeline triggering in v2.

Docs on pipeline triggering: https://circleci.com/docs/api/v2/#trigger-a-new-pipeline

Docs on pipeline parameters: https://circleci.com/docs/2.0/pipeline-variables/

[pixelastic]

Hello,

Thanks for the explanation and letting us know how things are moving, appreciate that.

I was previously triggering a specific job through the v1.1 API and was confused as why jobs triggered that way didn’t have a name in the UI (as can be seen from this Twitter exchange). I guess I didn’t really understand the difference between jobs, workflows and pipelines until then. Things are much clearer now.

I migrated my code to use the new v2 API and the parameter syntax. It took a bit of trial and error to understand how to specify which workflow I want to run… and it turns out it is not possible to specify it directly from the call. AFAICT all workflows are run if I manually trigger my pipeline from the API.

The solution I pieced together from your documentation and various comments on this forum is to add a when clause to each workflow and use parameters to flag which workflow I actually want to run. Below is an abbreviated config file, could you let me know if I’m on the right track or if you see any way to improve it?

version: 2.1

jobs:
  # omitted for brievity

parameters:
  workflow_commit:
    type: boolean
    default: true
  workflow_automatedRelease:
    type: boolean
    default: false

workflows:
  version: 2
  # Build/Test/Lint on each commit
  commit:
    when: << pipeline.parameters.workflow_commit >>
    jobs:
      - ci
  # Run every week
  weeklyUpdate:
    triggers:
      - schedule:
          cron: '0 9 * * 2'
          filters:
            branches:
              only:
                - master
    jobs:
      - weeklyUpdate
  # Run only when manually triggered by the API
  automatedRelease:
    when: << pipeline.parameters.workflow_automatedRelease >>
    jobs:
      - automatedRelease

And then, the curl call that explicitly turns off the commit workflows and turns on the automatedRelease one. The weeklyUpdate workflow being on a trigger, it seems that I don’t need to explicitly turn it off, is that correct?

curl -u ${CIRCLECI_TOKEN}: -X POST --header "Content-Type: application/json" -d '{
  "parameters": {
    "workflow_commit": false,
    "workflow_automatedRelease": true
  }
}' https://circleci.com/api/v2/project/github/pixelastic/pathfinder-society/pipeline

This seems to correctly trigger only the automatedRelease workflow when used. Could you confirm this is the recommended way to trigger a specific workflow using the v2 API?

[Brian-Triplett]

We started with CircleCI back in the 1.0 days. While we migrated all our yml to 2.0 we still have a number of apps that are job-based and use this API to trigger. Could there be any relief on the timeline as it will take us a while to convert them all over? Any help would be appreciated.

[ndintenfass]

Do you have account manager? If so, best to go through them. If not, do you know how many projects you need to change?

[Brian-Triplett]

Yes we do and I’m going to email her as well.

We have 6 apps (not necessarily projects/repos in the CircleCI sense) that are built around the https://circleci.com/api/v1.1/project/github/{org}/{repository}/tree/{branch} endpoint being able to pass a CIRCLE_JOB parameter to trigger a specific CircleCI job. Basically we have build pipeline automation where something happens on a schedule or external trigger (not in Github) that triggers a job in CircleCI. Since we’ve been with CircleCI for 3 years we’d invested a lot of our pipeline automation tooling in the v1 API.

We also use parameters/environment variables quite a bit as well and the changing parameter handling is also pretty impactful.

We completely understand the benefits and the need to move forward but we’ll have to change a fair amount of code to move to API version 2. Ideally we’d love to update everything to use 2 but that will be impossible to do by March 1. It’s not just changing yml for us but redesigning some of the pipeline automation we’ve built to work with the new API version and new parameter handling.

[ndintenfass]

Thanks Brian, this is super-helpful detail. We have been experimenting with a stop-gap approach where all API v1.1 job triggers are re-routed through our pipelines machinery. If that works we will likely create some grace period and can work with you directly on a transition plan.

Have you already done a direct assessment of the migration cost? Our goal is that when changes are necessary to move from the old approach to the new approach for triggering work that the cost will be “only” in config tweaks. In most cases, configs are fairly “small”, and the changes are minimal. But, if you’re finding that the changes you’d need to make will be fairly onerous I’d love to understand and see if there’s things we could do on our end to ease that pain.

[Brian-Triplett]

Thanks Nathan. I appreciate the understanding. What’s got us is that we’ve built other business apps that trigger Circle jobs via the 1.1 API from other places (AWS, Slack, Salesforce). After thinking about it more I think the real questions is how we would reach in and trigger a specific job.

We have a few projects that look like this:

jobs:
  build:
    <build steps here>

  deploy:
    <deploy steps here>

We then use the endpoint and parameters I mentioned above to trigger one of the jobs (like deploy) from an external service. Is there going to be a replacement to that? What would be the recommended approach in the new API? Would we convert to workflows? I didn’t see a way to trigger a workflow in the API documentation.

[ndintenfass]

We would prefer that move to workflows, yes. We recently added a when parameter under each workflow declaration that can be used in conjunction with pipeline variables to dynamically run certain workflows based on boolean variables (including parameters you can define for the pipeline). We are also considering adding similar functionality to individual jobs within a workflow, but for now it’s not clear we need to add that complication - we do invite feedback, of course.

When we built the when clause we wanted to provide a more general-purpose semantics around how a person writing config could manage what to do based on certain inputs (parameters explicitly declared for the pipeline) or based on the metadata associated with the pipeline (today we only expose the pipeline ID and number this way, but we hope to add trigger information, so you can have dynamism based on, for instance, whether it’s a “Draft PR”). We decided that the names of workflows is an internal concern of the config, and that a config author should not need to worry about whether or not an API client triggering pipelines knows about the internal structure of config beyond the declare parameters. We felt this was a cleaner contract and a more deterministic interface with less chance of unforeseeable breakage (ie: changing the shape of explicit inputs is different in kind from changing the arbitrary name given to a workflow that is run). Thus, rather than make job name or workflow name the input, we let the config author declare specific inputs (along with type and default values) and use that to drive the dynamism based on what is sent with the API POST request to trigger work.

You can see examples of using when under a workflow here: https://circleci.com/docs/2.0/pipeline-variables/#conditional-workflows

Assuming you’ve been running your API-triggered jobs regularly this week, it’s likely that you ran through our code that now automatically runs them through the pipelines machinery, which will automatically wrap workflows around them anyway (you can check this by seeing if those API-triggered runs show up on the workflows list (or on the Pipelines list in our new UI now in preview).

[okcoleman]

I’m in the middle of transitioning a job-triggering strategy to a pipeline-triggering strategy.

It took me a while to figure out how to declare variables.

Note: I understand that I need to change the env var references in my “command” to the new << >> params syntax. My question is: why doesn’t the UI show my pipeline triggering?

I’m testing this on a branch: new_circle_config

version: 2.1
parameters:
  # We must declare parameters that we intend to POST from the API.
  # https://circleci.com/docs/2.0/reusing-config/#parameter-syntax
  #
  # We specify dummy sentinal values for the defaults. The API seems to complain
  # if there are no defaults, but we really need to have these explicitly passed
  # by our caller in every case. Instead, we check for "xxx" and bail if we see
  # that in any parameter.
  # https://circleci.com/gh/paperlesspost/kubernetes-deployments/131
  NAMESPACE:
    type: string
    default: "xxx"
  SERVICE:
    type: string
    default: "xxx"
  VALUE:
    type: string
    default: "xxx"
  VALUES_PATH:
    type: string
    default: "xxx"

jobs:
  argo-release:
    docker:
      - image: "quay.io/paperlesspost/argo-repo:latest"
        auth:
          username: ${QUAY_USER}
          password: ${QUAY_PASSWORD}
    steps:
      - add_ssh_keys:
          fingerprints:
            - "7a:55:30:73:dd:0e:75:1a:82:e4:d8:a9:c3:e1:7c:5f"
      - checkout
      - run:
          name: Commit changes to kubernetes-deployments
          command: |
              [[ $NAMESPACE == "xxx" ]] && echo "must provide a NAMESPACE parameter!" && exit 1
              [[ $SERVICE == "xxx" ]] && echo "must provide a SERVICE parameter!" && exit 1
              [[ $VALUE == "xxx" ]] && echo "must provide a VALUE parameter!" && exit 1
              [[ $VALUES_PATH == "xxx" ]] && echo "must provide a VALUES_PATH parameter!" && exit 1
              argo-repo set --checkout-dir $(pwd) \
                --namespace $NAMESPACE  --service $SERVICE \
                --values-path $VALUES_PATH --value $VALUE
  promote-from-namespace:
    docker:
      - image: "quay.io/paperlesspost/argo-repo:latest"
        auth:
          username: ${QUAY_USER}
          password: ${QUAY_PASSWORD}
    steps:
      - add_ssh_keys:
          fingerprints:
            - "7a:55:30:73:dd:0e:75:1a:82:e4:d8:a9:c3:e1:7c:5f"
      - checkout
      - run:
          name: Get a value from an existing namespace and set it in another
          command: |
            [[ $NAMESPACE == "xxx" ]] && echo "must provide a NAMESPACE parameter!" && exit 1
            [[ $SERVICE == "xxx" ]] && echo "must provide a SERVICE parameter!" && exit 1
            TAG=$(argo-repo --checkout-dir $(pwd) get --namespace $FROM_NAMESPACE --service $SERVICE --values-path pp-service.image.tag)
            echo "Copying existing tag $TAG from namespace $FROM_NAMESPACE to namespace $TO_NAMESPACE"
            argo-repo set --checkout-dir $(pwd) --namespace $TO_NAMESPACE --service $SERVICE --values-path pp-service.image.tag --value $TAG
workflows:
  version: 2
  workflow:
    jobs:
      - argo-release:
          filters:
            branches:
              ignore: /.*/

I trigger with this script:

#!/bin/bash

curl -X POST \
  -u ${CIRCLE_API_TOKEN}: \
  --header 'Content-Type: application/json' \
  -d '{
  "branch": "new_circle_config",
  "parameters": {
      "NAMESPACE": "development",
      "SERVICE": "chartmuseum",
      "VALUE": "55Mi",
      "VALUES_PATH": "pp-service.resources.limits.memory"
  }
}' https://circleci.com/api/v2/project/gh/paperlesspost/kubernetes-deployments/pipeline

The API responds with a new “pending” job.

The UI adds a new blank line, but shows nothing else to me. It doesn’t look right.

[pdhnai]

With the v2 pipeline triggering API the UI no longer displays the commit message (also discussed in this related post)

It’s also unclear to me how and what the attribution headers are used for. Setting x-attribution-login header in the request doesn’t seem to have an effect on the resulting pipeline attribution.

[gempain]

My only concern for moving to v2 is that you only allow using a personal access token (full read/write access) instead of the read-only project api token. This is a security issue IMO. At least, we should have a choice.

[amuslija]

With the deprecation of the endpoint for triggering individual jobs in API 1.1, my team is forced to move to triggering pipelines.

However, this is not working as expected, as triggering pipelines with tag results in workflows not being run (basically the pipeline gets created, but no workflows). This issue is very similar to the one that @okcoleman described.

When providing the branch name instead of the tag name when triggering a pipeline, workflows do get created, which makes things even weirder.

I thought that my CircleCI config file had some bad syntax that slipped and was not reported, but I created a branch (and tagged the commit) with a config file taken from the Getting Stared guides:

version: 2.1
jobs:
  build:
    docker: 
      - image: circleci/node:4.8.2 # the primary container, where your job's commands are run
    steps:
      - checkout # check out the code in the project directory
      - run: echo "hello world" # run the `echo` command
workflows:
  version: 2
  build_and_test:
    jobs:
      - build

And tried to trigger a pipeline with the standard REST POST request I found in the API v2 docs:

curl -X POST https://circleci.com/api/v2/project/gh/AulaEducation/aula/pipeline?circle-token=${CIRCLECI_TOKEN} \
  -d '{ "tag": "v33-pipeline-test"}' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -H 'x-attribution-login: string' \
  -H 'x-attribution-actor-id: string'

The workflows/jobs still do not get created and the GET api/v2/pipeline/${pipelineInfo.id}/workflow returns an empty list of items. Opening the Pipelines web page, we see the same blank lines that @okcoleman posted.

Considering that the build job endpoint is being deprecated, and that triggering pipelines when providing tags is not working as expected, my team is currently facing a broken CI/CD process.

We contacted the support (not the premium one though) to get information about what a potential issue could be, and we barely got any useful feedback back. This has been going on for more than 10 days. I am a bit shocked that the hello world-type example from the API v2 docs is not working as expected.

We are optimistic that these issues will be resolved, and I personally think that the new API as well as the new version of config file, allow more flexibility when designing a CI/CD process, but we will be forced to look for another vendor if our basic workflow is not working at all.

[bgneto]

@ndintenfass

I also hope to solicit input on whether anyone reading this feels like they need more time than March 2020 to stop using the job triggering in v1.1 and switch to pipeline triggering in v2.

Without the “tag” option working properly or having a new option to send the “revision” that needs to be triggered I don’t see how we can migrate do v2 until the end of this month.

[bgneto]

Update: I got the tag option API call to work as expected. My mistake was not adding the filter to the workflow/job.
Never the less an option to trigger the pipeline with a given revision is must appreciated.

[amuslija]

@bgneto

Update: I got the tag option API call to work as expected. My mistake was not adding the filter to the workflow/job.
Never the less an option to trigger the pipeline with a given revision is must appreciated.

This helped me out so much! It’s weird that this details is missing from the Config/API documentation.

[dcosgrove]

Is this March 1st 2020 EOL still expected, or is that postponed?

I’m a bit surprised if so, given that all of the documentation and guides(see below for examples) on the site are still suggesting usage of the v1.1 API.

Is there any way that this date can be pushed back, at least until the documentation has been updated to reflect usage of the new APIs?

[dcbachi]

For what it’s worth, I set out today on a simple mission to “trigger a CircleCI job via the API”, and after a lot of contradictory documentation and trial-and-error, reading this thread is the closest I’ve come to figuring out what’s going on.

This is the first thing I came across via Google, which doesn’t explain how to actually trigger a single job.

This is the first thing I found that points out that I can use build_parameters[CIRCLE_JOB] to do this. Unfortunately, the example there uses Version 2 CI config, and according to the first doc I linked, these endpoints won’t work with Version 2.1 config (Who would think a minor version upgrade would remove API support?)

Eventually, I discovered the v2 API and started working on that, but, before I could get it working, I somehow discovered that the v1.1 API actually works fine with my Version 2.1 config (which I imagine has something to do with the backwards-compatibility workaround mentioned in this post?)

With a lot of the v2 API docs I can find marked “preview”, I’m still not sure what I’m actually supposed to be using.

Overall, this has been a disappointing experience.

[sulami]

We’re sorry about the confusing transition period.

V1.1 job triggering does indeed work right now, but is going to stop working in the future, as it’s fundamentally incompatible with pipelines. The current compatability layer is very brittle as a consequence.

We are actively working on expanding 2.1 config and the v2 pipelines API to add ways to trigger jobs. I’m currently working on releasing compile-time config, which will lay the foundation for a lot of these capabilities.

Here is an example of selecting a workflow via the API using pipeline parameters.

Until we have enabled all the v.1.1 use cases in v2, we will have to keep the v1.1 docs around, which is understandably confusing, but we’re hoping to get to a point soon where we can just get everyone to use v2 and deprecate v1.1

Let us know if there is something in particular you’re missing, and we’ll try to build something to fix that.

Wrt. the APIv2 docs being marked preview, most of them should be stable now, and we should remove the preview designation. I don’t expect any breaking changes to any of the core API endpoints (pipelines/workflows) at this stage.

[philglass]

@sulami do you have any more concrete timelines on when the v1 API will go away? It’s going to be quite a lot of work for us to migrate, and we’re unlikely to prioritise this until it’s strictly necessary (because v1 does everything we need).

[stig]

No; we have not yet set a date for v1 API job triggering going away. We’ll be sure to update this thread when we do.