Overriding Go Inference in the Dependencies Phase

Build Environment Programming Languages
, ,
1

If you’d like to add commands to the dependencies phase of your go build, the easiest way to do this is to add them in the dependencies: post: section of your circle.yml. However, many users need to override the inferred go get and go build steps.

Overriding Go inference in the dependencies phase is known to break builds. If you’re familiar with CircleCI, you should know that your repo is cloned into ~/$CIRCLE_PROJECT_REPONAME, which is not in the GOPATH. This would normally mean that some go commands executed from this directory will fail, since they must be executed from the GOPATH. Our inference does some magic to make things work, but overriding inference will also override the magic.

In case you’re curious about the "magic, we do the following:

  1. Create a $HOME/.go_project/src/github.com/$CIRCLE_PROJECT_USERNAME directory.
  2. Create a symlink in the newly created directory to the default checkout directory ($HOME/$CIRCLE_REPO_NAME).
  3. Add the newly created symlink to the GOPATH.
  4. Change the build_dir to the path of the newly created symlink so that subsequent commands are executed from this directory.

The good news is that you can (almost) replicate this magic yourself with the power of your circle.yml.

The end result should look something like this:

dependencies:
  override:
    - mkdir -p $HOME/.go_project/src/github.com/$CIRCLE_PROJECT_USERNAME
    - ln -fs $HOME/$CIRCLE_PROJECT_REPONAME $HOME/.go_project/src/github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME
    - |
      echo 'export GOPATH=$GOPATH:$HOME/.go_project' >> ~/.circlerc

Note: Changing the build_dir (or any circle.yml config) is not possible in the middle of a build, so subsequent commands that must be executed from the GOPATH must be prefixed with cd ~/$HOME/.go_project/src/github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME && . For example, go get would look like this:

    - cd $HOME/.go_project/src/github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME && go get -t -d -v ./...

​
Happy Testing!

2 Likes
3

Here’s how i implemented this idea:

machine:                                                                                                                                                                                                       
    environment:
      BASE_DIR: $HOME/.go_project/src/github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME

general:
  build_dir:
    ../../../$BASE_DIR

dependencies:
    override:
        - mkdir -p $(dirname $BASE_DIR) && ln -fs $HOME/$CIRCLE_PROJECT_REPONAME $BASE_DIR:
            pwd: /
        - echo 'export GOPATH=$GOPATH:$HOME/.go_project' >> ~/.circlerc

So far it seems to be working nicely. Thanks!

3 Likes
4

I had some real trouble with this not working for branches and not being easy to maintain. Instead, I came up with a neater approach:

machine:
  environment:
    GO_REPO_PATH: $HOME/.go_workspace/src/github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME

dependencies:
  override:
    - go get -t -d ./...
    # We have to delete the source for this repo that 'go get' checks out, as it always fetches master.
    - rm -rf $GO_REPO_PATH
    # Then replace it with the version checked out by Circle CI, which may be a branch
    - ln -s ~/$CIRCLE_PROJECT_REPONAME $GO_REPO_PATH
test:
  override:
    - go test ./...

I let go get clone the master branch of the repository itself into .go_workspace, but then delete that and replace it with the code checked out by circle by sym-linking it.
Hope someone else finds it useful…