We are doing something fairly unique in regards to what we are testing, which is doing integration-level tests for a solution that configures an entire cloud (i.e. an instance of our infrastructure at a DC) via a single CLI execution (composes Ansible at a lower layer and exposes a simple interface via a bash-based CLI). The workflow to perform these integration tests looks like:
- Install Tugboat, a CLI for DigitalOcean, and Ansible (
- Create a ~16 node/droplet cloud on DigitalOcean via Tugboat (?, right now
- Run Ansible site.yml which can be thought of as a test itself, exiting 0 on success and 1 on failure (
- Run other integration-like Ansible playbooks which test for proper cloud state, exiting 0 on success and 1 on failure (
- Destroy the ~16 node/droplet cloud on DigitalOcean via Tugboat (?, right now
With all that in mind, a few pointed questions on the phases/sections and their lifecycles:
- Right now, I have the cloud creation step in
dependencies.prebecause it has a desirable feature in that if it fails it will not run the test phase/section.
test.predoes not have this characteristic while
database.predoes. Is there a better spot for this?
- Right now, I have the cloud destruction step in the undocumented
teardown.post, however it does not run in certain circumstances leaving the cloud up. For instance, if
dependencies.prefails, for whatever reason, this phase/section will not run. Are there any phases/sections and lifecycles steps that exhibit the characteristic of always running, no matter which previous phases/sections failed? Kinda like a finally block.