RFC: Our ES6+Node+Rethink+Dokku Build+Deploy Configuration

nodejs
codedeploy

#1

This :snowflake:️snowy​:snowflake:️ Saturday here in New York was the perfect time to dig into CircleCI 2.0. We had been having difficulties with caching the node_modules with the current production version, so we were eager to give something new a try.

First impression: we love this new version.

It was easy to get started. The YAML configuration is straight forward and flexible. Being able to specify docker containers to compose our own testing environment feels like the future (and closely matches how we run our own operations). The way caching is handled is simply brilliant (look at how wonderfully it works with Yarn)!

We’d love some comments on our configuration and if there are any suggestions for how to improve it. First some notes on our application and who we are: we’re a small pre-seed-stage startup building a SaaS teaming application. Our application, Action, is open source. Our application is ES6/React on the frontend and Node+Rethink on the backend deploying to servers Digital Ocean using dokku.

Our build and deployment process occur in three steps:

  1. Webpack builds a prerender.js bundle for the server
  2. Webpack builds a minified and chunked client assets and pushes them to AWS S3
  3. The repository, including the build directory, is deployed using dokku to the server infrastructure via pushing to special github remote

When using CircleCI 2.0, in order to fetch the S3 configuration, we’ve added an ssh key for a github machine user. The ssh key is used to check out a special private repository and the environment variables are copied from the repository (an alternative is that we store them on CircleCI, but this seemed more convenient…)

Here’s the active pull request to update our CircleCI configuration.

And here’s just the config we’re trying:

version: 2
executorType: docker
containerInfo:
  - image: node:6.9.4
  - image: rethinkdb:2.3.5
stages:
  build:
    workDir: /home/ubuntu/action
    environment:
      - DEVOPS_REPO: "git@github.com:ParabolInc/action-devops.git"
      - DEVOPS_WORKDIR: "/home/ubuntu/action-devops"
      - GITHUB_REMOTE_PRODUCTION: "dokku@action-production.parabol.co:web"
      - GITHUB_REMOTE_STAGING: "dokku@action-staging.parabol.co:web"

    steps:
      - type: add-ssh-keys
        fingerprints:
          - "53:a8:37:35:c3:7e:54:f5:19:f6:8e:a1:e0:78:52:da"

      - type: checkout

      - type: shell
        name: Switch to Github user key
        command: |
          cp -f $(ls -1 ~/.ssh/id_rsa_* | head -n 1) ~/.ssh/id_rsa
          rm -f ~/.ssh/*.pub

      - type: cache-restore
        key: action-{{ checksum "yarn.lock" }}

      - type: shell
        name: Install dependencies
        command: |
          apt-get update
          apt-get -yq install build-essential g++
          npm i -g yarn
          yarn

      - type: cache-save
        key: action-{{ checksum "yarn.lock" }}
        paths:
          - /home/ubuntu/action/node_modules

      - type: shell
        name: Database migration
        command: npm run db:migrate

      - type: shell
        name: NPM lint
        command: npm run lint

      - type: shell
        name: NPM run test
        command: npm test

      - type: shell
        name: Pre-build DevOps checkout
        command: |
          if [ "${CIRCLE_BRANCH}" == "production" ]; then
            export DEPLOY_ENV="production";
          elif [ "${CIRCLE_BRANCH}" == "staging" ]; then
            export DEPLOY_ENV="staging";
          else
            export DEPLOY_ENV="development";
          fi
          git clone $DEVOPS_REPO $DEVOPS_WORKDIR &&
          cp $DEVOPS_WORKDIR/environments/$DEPLOY_ENV ./.env

      - type: shell
        name: Build test
        command: npm run build

      - type: deploy
        name: Possible deployment
        command: |
          if [ "${CIRCLE_BRANCH}" == "production" ]; then
            export GITHUB_REMOTE="${GITHUB_REMOTE_PRODUCTION}";
          elif [ "${CIRCLE_BRANCH}" == "staging" ]; then
            export GITHUB_REMOTE="${GITHUB_REMOTE_STAGING}";
          fi
          if [ -n "${GITHUB_REMOTE}" ]; then
            npm run build:deploy &&
            git add build &&
            export ACTION_VERSION="\
              $([[ $(grep version package.json) =~ (([0-9]+\.?){3}) ]] && \
                echo ${BASH_REMATCH[1]})"
            git config --global user.name "Parabol CircleCI"
            git config --global user.email "admin+circleci@parabol.co"
            git add build
            git commit -m "build $ACTION_VERSION" build
            git remote add dokku $GITHUB_REMOTE
            ssh -o StrictHostKeyChecking=no \
              $(echo $GITHUB_REMOTE | cut -f1 -d:) -T > /dev/null &&
            git push -f dokku $CIRCLE_BRANCH:master
          fi

      - type: artifacts-store
        path: /home/ubuntu/action/build
        destination: build

Anything jump out at you that we should change?


#2

Thanks for all the detail! One thing that I have noticed lately is that when you run multiple shell commands only the last one’s exit code will fail a build.

So for instance, if npm i -g yarn returns a non zero exit code it will still keep going which could result in some non-deterministic behavior.

My suggestion would be to put these into a script and then call that script rather than running multiple different commands in a single step. This way, if any of the steps fail in the script then the exit code will be the proper one and the build will fail or succeed accordingly.


#3

Instead of rolling these into a separate bash script, you can also use set -e in each section.

Compare these two steps and their output:

stages:
  build:
    workDir: ~/my-project
    steps:
      - type: checkout
      - type: shell
        command: |
          ls asdf
          ls fdsa
ls asdf
ls fdsa
ls: cannot access asdf: No such file or directory
ls: cannot access fdsa: No such file or directory
Exited with code 2

And with set -e:

stages:
  build:
    workDir: ~/my-project
    steps:
      - type: checkout
      - type: shell
        command: |
          set -e
          ls asdf
          ls fdsa
set -e
ls asdf
ls fdsa
ls: cannot access asdf: No such file or directory
Exited with code 2

#4

Thanks @Eric, this is a much cleaner solution IMO.


#5

Here’s how things have evolved over the past few days. (link to below circle.yml on our repository)

version: 2
executorType: docker
containerInfo:
  - image: node:6.9.4
  - image: rethinkdb:2.3.5
stages:
  build:
    workDir: /home/ubuntu/action
    environment:
      - DEVOPS_REPO: "git@github.com:ParabolInc/action-devops.git"
      - DEVOPS_WORKDIR: "/home/ubuntu/action-devops"
      - GITHUB_REMOTE_PRODUCTION: "dokku@action-production.parabol.co:web"
      - GITHUB_REMOTE_STAGING: "dokku@action-staging.parabol.co:web"
      - PRODUCTION_BACKUP_VOLUME: "/mnt/volume-nyc1-01/action-production"

    steps:
      - type: add-ssh-keys
        fingerprints:
          - "53:a8:37:35:c3:7e:54:f5:19:f6:8e:a1:e0:78:52:da"

      - type: checkout

      - type: shell
        name: Switch to Github user key
        command: |
          cp -f $(ls -1 ~/.ssh/id_rsa_* | head -n 1) ~/.ssh/id_rsa
          rm -f ~/.ssh/*.pub

      - type: cache-restore
        key: action-{{ checksum "yarn.lock" }}

      - type: shell
        name: Install dependencies
        command: |
          apt-get update &&
          apt-get -yq install build-essential g++ &&
          npm i -g yarn &&
          yarn

      - type: cache-save
        key: action-{{ checksum "yarn.lock" }}
        paths:
          - /home/ubuntu/action/node_modules

      - type: shell
        name: Database migration
        command: npm run db:migrate

      - type: shell
        name: NPM lint
        command: npm run lint

      - type: shell
        name: NPM run test
        command: npm test

      - type: shell
        name: Pre-build DevOps checkout
        command: |
          if [ "${CIRCLE_BRANCH}" == "production" ]; then
            export DEPLOY_ENV="production"
          elif [ "${CIRCLE_BRANCH}" == "staging" ]; then
            export DEPLOY_ENV="staging"
          else
            export DEPLOY_ENV="development"
          fi
          git clone $DEVOPS_REPO $DEVOPS_WORKDIR &&
          cp $DEVOPS_WORKDIR/environments/$DEPLOY_ENV ./.env

      - type: shell
        name: Build test
        command: npm run build

      - type: deploy
        name: Possible deployment build
        command: |
          if [ "${CIRCLE_BRANCH}" == "production" ]; then
            export GITHUB_REMOTE="${GITHUB_REMOTE_PRODUCTION}"
          elif [ "${CIRCLE_BRANCH}" == "staging" ]; then
            export GITHUB_REMOTE="${GITHUB_REMOTE_STAGING}"
          fi
          if [ -n "${GITHUB_REMOTE}" ]; then
            npm run build:deploy &&
            export ACTION_VERSION="\
              $([[ $(grep version package.json) =~ (([0-9]+\.?){3}) ]] && \
                echo ${BASH_REMATCH[1]})"
            git config --global user.name "Parabol CircleCI"
            git config --global user.email "admin+circleci@parabol.co"
            git add build
            git commit -m "build $ACTION_VERSION" build
          fi

      - type: deploy
        name: Possible (production backup) and deployment
        command: |
          if [ "${CIRCLE_BRANCH}" == "production" ]; then
            export GITHUB_REMOTE="${GITHUB_REMOTE_PRODUCTION}"
          elif [ "${CIRCLE_BRANCH}" == "staging" ]; then
            export GITHUB_REMOTE="${GITHUB_REMOTE_STAGING}"
          fi
          if [ -n "${GITHUB_REMOTE}" ]; then
            git remote add dokku $GITHUB_REMOTE
            export SSH_DESTINATION=$(echo $GITHUB_REMOTE | cut -f1 -d:)
            ssh -o StrictHostKeyChecking=no "${SSH_DESTINATION}" -T >/dev/null
          fi &&
          if [ "${GITHUB_REMOTE}" == "${GITHUB_REMOTE_PRODUCTION}" ]; then
            $DEVOPS_WORKDIR/dokku/rethinkdb-backup.sh \
              -s "${SSH_DESTINATION}" -d "${PRODUCTION_BACKUP_VOLUME}"
          fi &&
          if [ -n "${GITHUB_REMOTE}" ]; then
            git push -f dokku $CIRCLE_BRANCH:master
          fi

      - type: artifacts-store
        path: /home/ubuntu/action/build
        destination: build

Some low-priority desires:

  • Use node container version from our package.json
  • Export environment variables between configuration sections (possibly via section environment: configuration)

#6