# Testing orbs

**URL:** <https://discuss.circleci.com/t/testing-orbs/26500>\
**Category:** Orbs\
**Created:** [November 14, 2018, 3:04am UTC](https://discuss.circleci.com/t/testing-orbs/26500 "2018-11-14T03:04:49Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![juanca](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/juanca/32/14598_2.png) [@juanca](https://discuss.circleci.com/u/juanca)\
**Post date:** [November 17, 2018, 2:47am UTC](https://discuss.circleci.com/t/testing-orbs/26500/4 "2018-11-17T02:47:45Z")

</div>

After two days of fiddling, here are my thoughts – note some of these can be seen throughout my [commit history in this PR](https://github.com/mavenlink/orbs/pull/3):

1. Schema validation

This seems to be the easiest way to automate some basic tests for validity. I would recommend checking out the `circleci/circleci-cli` and `circleci/orb-tools` orbs.

1. Expansion testing

The concept is pretty straight-forward. Essentially, it validates the schema of a configuration that utilizes the orb under test. However, putting it in an automated CI pipeline requires a bit of work due to CLI limitations. I might be able to simplify the work by either (a) utilizing the `circleci config pack` functionality, (b) referencing a local orb file, or © `circleci config process`ing a file the exact same way the “build processing” feature works.

I haven’t used this kind of test to assert on the structure of the resulting YAML file. As of this writing, I am favoring end-to-end tests because my `git` orb uses an external CLI and I do not see value in asserting string patterns.

1. Runtime testing

This seems to be the cake for my use-case. I would like to make a build with a few different jobs (that test the commands, jobs, and parameters of my orb). Ideally, my assertions would test the output of some steps – essentially printing the state of the machine and asserting some values.

I think a few things should be noted for this section. This is copy pasta from my [other thread](https://discuss.circleci.com/t/programmatically-building-a-new-job-with-a-new-config/26572):

1. Job must be a `machine` executor – otherwise the docker daemon is not properly setup.
2. You must install the latest `circleci` CLI – easiest way to do that is by using the `circleci/circleci-cli` orb.
3. Make steps that executes the build with a processed configuration file:

There is probably a way to get this workin with `orb-tools` but I wasn’t able to grok it initially and went with hand rolling my own implementation for the time being. I will try to refactor back into `orb-tools`.

While this approach provides me with the most value, I realized the assertions and messaging patterns are not obvious. It would be nice to see some improvements on the syntax (perhaps a command?) which allows for flexible assertions and error messaging.

```auto
  steps:
  - git/checkout # which is my orb under test
  - verify:
      actual: git log --pretty=form:'' | wc -l
      expected: 1
      message: Expected git history to be truncated to 1 commit but got

```

1. Integration testing

I have not a use case for this… yet.

---

_[View the full topic](https://discuss.circleci.com/t/testing-orbs/26500)._