Globbing tests does not include subfolders


We noticed today during our upgrade from 1.0 to 2.0 that globbing tests does not include tests that are in subfolders.

Running the following in CircleCI gives the following count of our files.

circleci tests glob test/**/*.rb | circleci tests split --show-counts

Read 354 filename(s)
Current container # 0. Assigning 89 filename(s) on this container

Adding a /**/ to the path above changes the results and the output for the files.

circleci tests glob test/**/**/*.rb | circleci tests split --show-counts

Read 307 filename(s)
Current container # 0. Assigning 77 filename(s) on this container

We are trying to recursively run all of our tests and we have a lot of tests that are deeply nested. Is there any other way to do this?

We are also running into this issue after migrating to 2.0. Seems like an obvious issue, any suggestions?

Interesting. Does circleci tests have a syntax where you can supply the list piped in, or in a file? (This does sound like an unintended change, just wondering how to get you running again in the short term).

See here:

you need to add shopt -s globstar to enable ** wildcard

How/where do I add ‘shopt -s globstar’? I’m using the ruby-orb and can’t figure out how to make it glob all my subdirectories.

You’ll want to add it before the circleci tests glob command is run. Here’s a redacted version of my file using a multiline command:

      - image: *ruby_image
      - checkout
      - run:
          name: Run tests
          command: |
            shopt -s globstar
            TEST_FILES="$(circleci tests glob ${SPEC_PATH})"
1 Like