Testing orbs

Ecosystem Orbs
2

Hi @juanca,

It is possible to test local orbs, but involves a bit of trickery. For initial development you can simply “inline” the orb in your config.yml - https://circleci.com/docs/2.0/creating-orbs/#creating-inline-orbs

For automated testing, we have some general guidance in our SDK repo, https://github.com/CircleCI-Public/config-preview-sdk/blob/987fd917522f5d5d9a2cae145878fc548e9ea79e/docs/orbs-testing.md

But the tooling is left open to authors. I’ve created an opinionated flow that works for my purposes, and that can be seen in this sample repo - https://github.com/eddiewebb/circleci-dmz-orb/blob/master/.circleci/config.yml

I chose BATS as the framework to drive my expansion testing, and use a number of helpful functions to enable the assembly of configuration, as well as assertions. https://github.com/eddiewebb/circleci-dmz-orb/blob/7ac0e063781d462878ae7e6c1477b3116ac4fd67/test/test_expansion.bats

The utilities convert the processed YAML to JSON so that I can run assertions on specific elements using JQ, https://github.com/eddiewebb/circleci-dmz-orb/blob/7ac0e063781d462878ae7e6c1477b3116ac4fd67/test/test_expansion.bats#L35,L40

  assert_jq_match '.jobs | length' 1  #only 1 job
  assert_jq_match '.jobs["build"].steps | length' 5  # it contains 5 steps
  assert_jq_match '.jobs["build"].steps[3].run.command' 'ssh -L 9001:104.154.89.105:80 -Nf ubuntu@ec2-18-191-19-150.us-east-2.compute.amazonaws.com || true'  #the 4th is my compiled command
  assert_jq_contains '.jobs["build"].steps[2].run.command' 'KEY_VALUE=`cat somefile`'
  assert_jq_contains '.jobs["build"].steps[2].run.command' 'KEY_VALUE=`echo "somefile"`'
  assert_jq_contains '.jobs["build"].steps[2].run.command' 'echo "ec2-18-191-19-150.us-east-2.compute.amazonaws.com ${KEY_VALUE}" >> ~/.ssh/known_hosts'

You can also run builds with the configuration locally using the CLI, or as part of your workflow using our orb tooling., https://github.com/CircleCI-Public/orb-tools-orb/blob/master/src/%40orb.yml#L107

That creates a job in the workflow that expects input configuration to be tested against the orb, and allows the output, status and container state to be inspected afterwards.

This is all still evolving, so we would love any feedback you have!

2 Likes