Allow specification of jobs in other files


#1

A lot of jobs are alike, especially when testing on multiple versions of an interpreter (an extremely common CI task). Consider allowing us to specify all or part of a job in a separate file in the .circleci directory, e.g.

.circleci/config.yml:

jobs:
  'python3.6':
    docker:
      - image: 'python:3.6'
    include: unit_tests.yml

.circleci/unit_tests.yml:

---
steps:
  - checkout
  - run:
      name: Install things.
      command: pip install .
  - run:
      name: Run unit tests.
      command: python setup.py test

This would allow the unit_tests.yml file to be used across multiple jobs (in this example, it can be used for all four current Python interpreters).

It would also cut down on the size of config.yml, which can get quite large even for relatively simple projects.

Thank you for your consideration.


#2

Hey @lukesneeringer , have you considered using YML anchors / repeated nodes in this case? This is what we do in our config.yml where we have multiple jobs with the same setup/steps/etc… e.g.

references:
  working_dir: &working_dir
    /tmp/app 

  ruby_image: &ruby_image
    circleci/ruby:2.4.1

  ruby_container_config: &ruby_container_config
    docker:
      - image: *ruby_image
    working_directory: *working_dir

.....

jobs:
  build:
    <<: *ruby_container_config
    steps:
      - checkout
     .....

We specify all our common stuff at the top in the references section and just re-use it wherever we need to.

Hope that helps!


#3

I actually do have this. Even with that, my config file is enormous, and there are a lot of places where that does not work well because it is a shallow merge.


#4