CircleCI Local CLI doesn't validate config file's workflows

If the workflows section of a CircleCI config isn’t valid, running circleci config validate -c .circleci/config.yml will still return that it’s valid. What’s worse is that the Builds section of CircleCI’s interface won’t show anything. It’s like your build doesn’t exist. If however, you go to the Workflows section it shows a nice icon that says Needs Setup with an extremely helpful message about what in the workflows section was wrong. However this isn’t clear and it took me 30-60 minutes to figure out that was the correct way to debug this…

Fortunately I found 2.0 Builds not working… which was open for a month without any activity from CircleCI staff, despite clearly demonstrating that there’s a documentation issue for this.

Either the validate functionality of the CircleCI Local CLI should be improved so that it returns the same extremely helpful message as the workflows section of the interface, or the documentation for it should be updated to warn people that it is really only validating half of the config, not the whole thing which the documentation purports to do.

1 Like

See also:

We are working to improve the CLI experience around config, including fixing this problem. It’s very much on our radar to make config validation less sad-making.

2 Likes

Hi Kurt,

I’m working on the project to validate config in a better way from the CLI at the moment. Would it be possible for you to share your config file that caused the issue with us? Would you email it to marc@circleci.com please?

It will allow us to make sure that the issues that you hit are handled with our new system.

Thanks

Marc

2 Likes

Thanks for letting us know. Just wanted to add that I think Circle CI is great, but the ability to test locally is the biggest pitfall. Since I use workflows heavily the only way to validate or test my config is to push a commit. Not ideal. Also the error that appears in circle CI is very sad-making. But I see you guys are aware of the problem, so I look forward to the update!

Running workflows locally is likely a non-starter, but our long-term plan here is to make it much easier to run builds from the CLI. So although it would not run locally it wouldn’t require a commit->push->browser->find new build flow that is admittedly not ideal. A new version of the CLI that is in testing now (https://github.com/CircleCI-Public/circleci-cli/blob/master/README.md#getting-started) will also have better validation of configuration, so you shouldn’t need to do any pushing to know if the config is at least well-formed.

Has there been any progress on this? Has this new validation script made it out of testing yet? It probably would have prevented Previously valid 2.0 config now breaking without any related changes reported in the changelog from breaking my deploy process.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.