How to test against multiple infrastructure parts

Build Environment
,
1

Hi!

I’m coming to Circle from Travis-CI, and I’m trying to sort out how to run a test against multiple ENV configurations. This is important because I want to ensure that the project works on both node v4 and node v6, and with both “real” redis, and a package called fakeredis.

On Travis, I would configure my travis.yml to look like the following:

language: node_js
node_js:
  - "4"
  - "6"
env:
  - "FAKEREDIS=true"
  - "FAKEREDIS=false"

script: npm test

This would run my suite 4 times, with the combinatorics of:

  • node 4 & fake=true
  • node 4 & fake=false
  • node 6 & fake=true
  • node 6 & fake=false

I cannot sort out how to break my test up in this way on Circle. The best I can come up with is to run my test 4 times, ie:

test:
  override:
    # TESTING NODE V4.X
    - nvm install 4 && nvm use 4
    - rm -rf node_modules && npm install
    - FAKEREDIS=true npm test
    - FAKEREDIS=false npm test

    # TESTING NODE V6.X
    - nvm install 6 && nvm use 6
    - rm -rf node_modules && npm install
    - FAKEREDIS=true npm test
    - FAKEREDIS=false npm test

This is obviously quite slow (and I can’t cache the node_modules directory)… and if one test fails, the whole thing fails.

Help?!

2

An example of this test suite on Travis can be found here: https://travis-ci.org/evantahler/actionhero/builds/147824723

3