We have been using yaml anchors in all of our config.ymls for about a year. Now we get a warning like this:
Your config file has errors and may not run correctly:
#/jobs: 88 schema violations found
#/jobs/build_app: 8 schema violations found
#/jobs/build_app: extraneous key [load_app_packages] is not permitted
#/jobs/build_app: extraneous key [load_artefact] is not permitted
#/jobs/build_app: extraneous key [load_repo] is not permitted
#/jobs/build_app: extraneous key [load_api_packages] is not permitted
#/jobs/build_app: extraneous key [deploy_deps] is not permitted
#/jobs/build_app: extraneous key [load_tf] is not permitted
#/jobs/build_app: extraneous key [load_app_build] is not permitted
#/jobs/build_app: extraneous key [load_bdd_packages] is not permitted
The workflow appears to still run as normal, so it is clear that the yaml anchors are still being interpreted correctly at run time.
The last time this particular file changed was 5 days ago and there were builds without this warning up until today.
We are doing some really heavy refactoring of how we process config at present, and adding these schema warnings is one of the first steps we are taking. We have big plans for making config more modular and re-usable.
Can you email me a link to your build please, to marc@ and I’ll take a look at the specific issue that your config is hitting?
Hi @marc. Out of interest, are YAML references something CircleCI plans to support long-term? I have advocated that people look into them several times in this forum, to help DRY up their config. However, I don’t want to do that if they might break in the future.
YAML anchors will continue to work - don’t worry.
What we are working on right now is to stop allowing new projects to build if they have “invalid config”. Invalid config is a little bit nebulous right now. Ideally we want to make any extra/undocumented YAML keys under workflows or under jobs be an error (for new projects only).
The goal here is to catch typos and bugs in config. We have a lot of users that spell “environment” and “resource_class” incorrectly right now, for example. Typos like these are silent failures at present. A build would run with these typos, but without the environment or resource class that has been specified. We want to make these hard errors with a clear pointer to the key in the YAML that is at fault.
The new config error warnings are a way for us to start highlighting such errors and gathering data on what the most common errors in config are.
We want to leave the top-level of config.yml open to allow for any extra definitions to be referenced using anchors. For most of our internal projects, for example, we declare the docker image at the top-level using an anchor, and reference it in each job, to DRY the config.
We are also working on a modular, composable system for config in addition to the schema checking. We want to allow you to define steps, jobs and workflows once, and re-use then across projects. If you’d like to know more about this, I’d love to chat - email marc@
That sounds great @marc. Happy to read and offer ideas about this, but would be good to do it on this forum, so others can chime in and/or see what’s cooking.
Assuming this thread is as good a place as any, one thing I’ve seen suggested in the recent past is a way of specifying a YAML sub-block but being able to replace part(s) of it. I’ve seen people want to use YAML references in such a case, but were unable to because a single key in the block needed changing. I don’t personally have a need for this, but I could see how this would be useful.
I’m also quite interested in the modular config. We’ve used a heavy amount of YAML anchors to dry things up in our config.yml as well and other (more friendly?) alternatives would be great.
Any update on this? It’s hard enough trying to get machine configurations running with the severe lack of official documentation on it. Now the main machine config I’ve been using is broken:
build:
machine:
python:
version: 3.6.1
This throws errors:
jobs: build: machine: 0 subschemas matched instead of one
jobs: build: machine: expected type: Boolean, found: Mapping
jobs: build: machine: extraneous key [python] is not permitted
So now I have to use pyenv to install 3.6.1 on every build, which is a pretty big time sink.