Problem splitting tests using Jest

Build Environment Running Tests
1

This is our config to split tests:

- &test
  name: Run Tests
  command: |
    TESTFILES=$(circleci tests glob "./packages/main/src/**/__tests__/*.(test|spec).(js|ts|tsx)" | circleci tests split)
    echo "All test files:"
    circleci tests glob "./packages/main/src/**/__tests__/*.(test|spec).(js|ts|tsx)" | xargs -n 1 echo
    echo "This job test files:"
    circleci tests glob "./packages/main/src/**/__tests__/*.(test|spec).(js|ts|tsx)" | circleci tests split | xargs -n 1 echo
    echo "Running tests:"
    JEST_JUNIT_OUTPUT="./test-results/jest/results.xml" yarn jest --maxWorkers=4 --coverage --forceExit --ci ${TESTFILES}
    bash <(curl -s https://codecov.io/bash) -t $CODECOV_UPLOAD_TOKEN

test:
    <<: *defaults
    parallelism: 2
    steps:
      - attach_workspace:
          at: ~/app
      - restore_cache: *restore-mongodb-binaries-cache
      - run: *copy_env
      - run: *test
      - restore_cache: *save-mongodb-binaries-cache
      - store_test_results:
          path: ./test-results

Use use the following docker to test

docker:
- image: entria/node-ci:0.1.12
- image: circleci/redis:4.0.9

The time to run in 1, 2 or 4 is the same for us

3

Hi, what do you mean by “The time to run in 1, 2 or 4 is the same for us”? Are you talking about timing of running parallel container or an issue with one of your commands?

Have you already sort out this problem?

4

Our glob was wrong

use this

TESTFILES=$(circleci tests glob “./packages//src//tests/*.spec.ts” | circleci tests split)

1 Like
5

Does anyone succeed to manage to have the output test results using parallelisation --outputFile test-results.json ?

For example if a job after theses tests need test-results.json how did you manage to have the whole results into one file

1 Like
7

I have the same issue right now.

8

I noticed you clicked like on the last comment, so I’m assuming that’s the issue you’re referring to. I think they’re describing the situation in this support article. Where you’ve run your tests in parallel containers and you need to combine them into a single results file. There’s no automatic way to do that as far as I know. You’ll need to do what the support article describes. Create a “collection” job that is connected to the parallel job through a workspace. When all of the parallel jobs are complete they will pass the test result files to the collection job through the workspace. Just make sure that you’re naming the file something different during the parallel run.

version: 2.1
orbs:
  python: circleci/python@1.2
jobs:
  build-and-test:
    parallelism: 5
    executor: my-executor
    steps:
      - checkout
      - run:
          name: Run tests
          command: |
            set -e
            TEST_FILES=$(circleci tests glob "openapi_server/**/test_*.py" | circleci tests split --split-by=timings)
            mkdir -p test-results
            pytest --verbose --junitxml=test-results/junit-${CIRCLE_NODE_INDEX}.xml $TEST_FILES
      -  persist_to_workspace:
           root: .
           paths:
             - test-results
  downstream:
    executor: my-executor
    steps:
      - attach_workspace:
          at: ./test-results
workflows:
  sample:
    jobs:
      - build-and-test
      - downstream:
          requires:
            - build-and-test

I didn’t actually test that config so you might need to rework it a bit. But you’ll notice within the test-output file name for pytest I’m using CIRCLE_NODE_INDEX like:

pytest --verbose --junitxml=test-results/junit-${CIRCLEI_NODE_INDEX}.xml $TEST_FILES.

This will make sure all the test file outputs have a unique name when they’re persisted to the workspace.

When the downstream job runs it will attach the workspace to a folder called test-results, and all of the different test output files from the parallel runs will be available.

./test-results
  junit-0.xml
  junit-1.xml
  junit-2.xml
  junit-3.xml
  junit-4.xml

From here you could combine them in some way if you need to, or process them individually. There’s no automatic way to combine them. But there are some tools available that might be able to help. Is that the result you’re looking for?

4 Likes