Dependency builds


#1

In classic “well-known build servers” it’s possible to have a build be a trigger for a dependant build, effectively setting up a dependency graph which cascades your projects.

I’m aware that a build can trigger another by undertaking a curl request to the API, but this has the dependency backwards as “child” builds need knowledge of projects which use them.

Can functionality be introduced where a trigger for a build can be another build completing?


Upstream and downstream projects like Jenkins
#2

If I’m understanding correctly…

test:
  post:
    - (code to launch parent build)

This code on child build is what you are looking for?


#3

I am aware of this, but the dependencies are back-to-front.

Child builds (or libraries) shouldn’t need knowledge of what will use them, it’s the responsibility of a parent project to declare the dependencies and create the dependency graph.

It may also be the situation that the author of a project which has a dependency doesn’t have access to the dependency source / build.


#4

My solution to this, was to do it within my main project :

  1. Iterate a list of dependencies
  2. Clone them into the project
  3. Look for a standard named bash script in each one and call it.
  4. The standard named script calls other bash scripts as needed

But this does not cascade, and it misses out on the circle.yml structure/process.

So I agree with this feature request, but . . .

In my case, the subprojects are packages that can be unit tested outside the parent. However, functional testing without the containing product is too hard to be worth the effort. This wouldn’t be the case for everyone, but is something to consider when designing a listener trigger as you propose,

You’d want the parent to start building, trigger the children, wait on their completion before importing their latest versions and only then continue with functional testing.

(PS : My code for the dependency iterator is public, if anyone’s interested, just say so and I’ll post the links.)


#5

I’d like to see a recommended solution for this. We have several API based services some of which have a dependency on the other. For example, our front facing server depends on our user service. When new changes are committed to user service I’d like to initiate tests on the front facing service. Using the circle API seems like a reasonable way to start a build on the front facing server. But having a simpler way to introduce this in circle.yml would be nice.


#6

I would also like to see a recommended solution for this. I have four modules that good published to an appengine app that have python and node builds for each.


#7

Currently, I have a pair of bash scripts to setup and another to run the tests from the child, which get run in deps and test: override of the parent.

But this duplicates / doesn’t make use of the circle.yml setup, and is brittle.

+1 for support or recommended solution.


#8

@martinhbramwell I’m interested in your solution. I’d like to do something similar.

I have a core project and many projects that depend on it.

Every time someone submits a pull request for the core project, I’d like to run the children projects pointing to the new version of the core project.


#9

Hi, @etagwerker, That was over a year ago, :unamused:. If I recall correctly, in the mean time, I learned about npm link, and now use that instead of pulling stuff in with shell scripts.


#10

I’d also love to see an official solution to this. In particular, it would be great to see the workflow feature extended to multiple builds.

In the mean time, I have stored our package dependency graph in a series of files across our packages in a second file in .circlei/. I’m planning to use the CircleCI REST API to trigger downstream builds after each successful build. However, there doesn’t seem to be a great way to deal with circular dependencies (except for me to cut the dependency graph stored in the files) because doesn’t seem to be a way to transit other information through the CircleCI API.


#11

Our hack to make this work is curl. I’d love to see something official I can configure.

- deploy:
      name: Build Studio
      command: |
        if [ "${CIRCLE_PR_NUMBER}" == "" ]; then
          curl -v -X POST https://circleci.com/api/v1.1/project/github/$CIRCLE_PROJECT_USERNAME/studio/tree/$CIRCLE_BRANCH?circle-token=$STUDIO_TOKEN
        fi

Here, the upstream build manually triggers the downstream build on success, assuming the same branch name, and ignoring pull request builds.


#12