I have a relatively simple Python application I would like to test, there’s just one catch. We are using a separate system, flyway, for our database migrations. In order to run the tests we must follow these steps:
- Create a PostgreSQL database.
- Run the migrate command using the flyway/flyway docker image.
- Using the python image, install our requirements
- Using the python image, run the tests.
I have looked high and low at all the official documentation and this community, and the only conclusion I can draw is that this very basic process is fundamentally incompatible with CircleCI.
I need to perform two sequential different steps within the same job, but each step uses a different executor. The first would use flyway and the rest would use python. CircleCI does not seem to allow this. All commands in all steps are performed using the first image listed for the job. Is there any way for me to tell CircleCI to use one image for step 1 and a different image for step 2 in the same job?
The other option would be to split this work up across multiple jobs. However, each job in CircleCI stands alone. Workspaces seem like a fine solution for persisting files across jobs. However, those files could only be preserved from the primary executing image. How can I possibly preserve an entire PostgreSQL database across jobs?
I thought of a very hacky solution, which is to make a dump of the database and use workspaces to move that dump across jobs. But even that won’t work because the flyway image can’t be used to make the dump, and we’re back to the first problem which is that any individual job must run all of its steps within one and only one image/executor.
What am I supposed to do here? Thanks.