Better way to define workflow stages?

workflow
config

#1

I define one job per each compiler version & build flavor that needs to be tested, like this: https://github.com/sociomantic-tsunami/turtle/blob/v8.x.x/.circleci/config.yml

It works pretty well as long as there is only one workflow stage and all builds run in parallel. However now I want to add additional post-test job and definition of the workflow because rather awkward:

workflows:
    version: 2
    all:
        jobs:
        - test-1
        - test-2
        - test-3
        - test-4
        - test-5
        - test-6
        - post-test:
            requires:
            - test-1
            - test-2
            - test-3
            - test-4
            - test-5
            - test-6

jobs:
  test-1:
  test-2:
  test-3:
  test-4:
  test-5:
  test-6:

All this long list of job names has to repeated again and again - once in job list, once inside a workflow and one time per every job in a next stage that depends on them. This isn’t nice to maintain at all.

I tried injecting this array in YAML anchors same as job object templates but looks like it is not supported by YAML parser used by CircleCI in any way.

Are there any better options to define this? Ideally I would simply want to define ordered test stages and not write individual test dependencies (like in GitLab CI or Travis CI).


#2

Hi there,

YAML anchors should definitely work here—please open a support ticket and send over the config syntax you’re trying so we can help you out. (Or you can reply to this thread and see if anyone else in the community can assist.)

It’s worth noting that you can also split your Workflows config into not just multiple jobs but multiple Workflows—see our config file for the circleci-images repo for a good example (we use YAML anchors there, as well).


#3

Thanks for the response!

Sadly, I don’t see how having multiple workflows can help here as it is not possible to have dependencies across workflows.

This what I would like to do ideally but it is not valid config syntax: https://github.com/sociomantic-tsunami/turtle/pull/30/commits/f1cf19c9b417632cce24fe42b085db87cc531ae7 . From what I have found so far it is not defined in YAML in general when one attempts to merge array elements with anchors.

One possibility would be to allow nested arrays:

 workflows:
     version: 2
     test:
         jobs:
         - *test-job-list
         - d2tag:
             requires: *test-job-list

… but that requires explicit support from CircleCI config processor to flatten such structure and currently does not work either.

YAML anchors should definitely work here—please open a support ticket and send over the config syntax you’re trying so we can help you out.

Sadly this is only available for paid customers and I won’t get a management approval to start a paid plan until all issues are resolved :frowning:


#4

Hi there,

My apologies for suggesting you open a ticket.

I believe what you’re looking to do should work fine. The config file you linked to looks a little wonky, though. Take a look at the config file I linked in my previous reply for an example of how to use YAML anchors in your config.

Thanks!


#5

I believe what you’re looking to do should work fine.

Well, I am afraid it doesn’t work right now. This isn’t a guess because I have tried using all mentioned anchor usage schemes and they are rejected by CircleCI because of invalid config syntax.

The config file you linked to looks a little wonky, though.

Can you please be a little more specific?

Take a look at the config file I linked in my previous reply for an example of how to use YAML anchors in your config.

Unless I have missed something, your linked config is different from kind of workflow I talk about. There is a long list of jobs all depending on one previous job - this is a simple and straightforward case. The problem I talk about in this thread is about inverse situation - having long list of jobs all being dependencies for several follow-up jobs.


#6

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.