Multiple YAML files for cross-platform projects and easier test matrix configuration

circle.yml

#1

This is a feature request for CircleCI to add support for multiple YAML files instead of just the one circle.yml file we have today. There are two reasons for this request:

  1. Better cross-platform support. For example, mobile developers need to test and deploy their iOS apps on OS X hosts, and usually want to use Linux to do the same for their Android apps. Currently CircleCI doesn’t let us run some tests on OS X and some on Linux; it’s all-or-nothing and developers are forced to split their repos, use two CI providers, or run all tests on OS X and spend a lot of money.
    Some previous requests for this feature: Support for Multiple OSs, How to use circleci for linux, xcode, ... in one project?.

  2. Easier test matrix configuration. The way CircleCI does parallelism with $CIRCLE_NODE_INDEX is kind of awkward sometimes. Sometimes I’d rather write a bunch of separate circle.yml files – one for each test configuration – and have CircleCI run each in a separate container. Some previous requests for this feature: Build matrix, Build matrix configuration for iOS projects.

Proposal

CircleCI would support a new directory called “circle.d”, which contains any number of YAML files that can be named anything, like “android-tests.yml”, “ios-tests.yml”, and “testflight-deploy.yml”. Each of these files would be treated almost the same way that “circle.yml” is treated today.

Parallelism: by default, CircleCI would set up one container for each YAML file. Some YAML files may want extra parallelism, so CircleCI would need to build a way for each YAML file to specify how many containers it wants. This is a nice-to-have, though; for v1 it’d be fine to have one container per YAML file since you could easily make more YAML files for more parallelism.

If CircleCI were to build this, I’d be interested in moving all of the React Native project’s tests to CircleCI. Currently they’re split between CircleCI and Travis CI. I’d also move more of my own company’s tests and deployment scripts to CircleCI if it were easier to set up separate test containers for our Android and iOS apps.

Thanks for listening!


Support Linux, Xcode ... in one project
#2

Sounds like just what we need for React Native! :slight_smile:


#3

As a PM, I’m hesitant to post without having something concrete to say (e.g., we’ll ship a fix tomorrow!) or to specifically answer a question. However, I wanted to post to provide a feedback loop.

I am very grateful to have thoughtful folks commenting on this forum and wanted to express my appreciation for the time and effort you put into this. Our product team looks at Discuss often and actively uses it for ideas and prioritization.

This post in particular is great because I know the problem is real and troubling & I found this proposal interesting. I will be talking to the team about it!


#4

any news on that? I’m pretty sure a lot of people are just waiting :slight_smile:


#5

@rishimkumar any newa on this feature please?


#6

Is there in the meanwhile anything more to say about feature request? We’d love to see anything near the aforementioned solution.
Cheers
SH


#7

I actually got around it by installing updating docker with a script somebody provided, then installing docker compose with python PIP and running my different projects inside different containers.

But the request was to be able to run different build per subfolder.

My project structure is as follow:
/ ->git root folder
/proxy/ -> a proxy project in scala
/frontend/ -> a frontend project in angular and nodeJs
/user/ -> a user service
/…/ -> other services

And each service has its own build and configuration but as of now we can’t run a complex build like this on Circle (without the docker work-around).


#8

This feature would be incredibly helpful. I’ve been spending a ton of bandwidth trying to work around the lack of this feature.


#9

any update on this?

This will be really helpful for us!


#10

Want to know the status as well, since I want to test Xamarin cross-platform apps.


#11

Would like to know the status for these, our project would benefit greatly from this feature as well.


#12

tl;dr: rather than hack a fix for this, we are moving to our new 2.0 configuration which is designed to make multi-environment tests (matrixed tests) much easier.

Our general approach to addressing this is in our 2.0 Platform, currently in Beta (https://circleci.com/beta-access/). In CircleCI 2.0 you are not limited to our original workflow that assumed a linear path through a single flow. Instead, you can configure many different jobs, each of which can run in a separate environment. As of this writing you configure that environment by choosing a public Docker image, but more will be coming on that front. We are also working on ways to build even more power into orchestrating the execution of jobs (along with executing many jobs simultaneously), which will make matrix support fairly easy. While we’re in Beta there are some features that haven’t yet ported over, and the configuration in 2.0 has a different format, so you’ll need to do a bit of up-front work to get things moved over, but we are excited to see what people do with the new power and flexibility of CircleCI 2.0.


#13