Can I merge parallel coverage reports?

Build Environment
,
1

I’m running a parallel test pass in CircleCI (RSpec, Ruby on Rails). This has been running fine, but now I want to also run a SonarCloud scan and get a coverage report (among other things). Right now I’m running SonarCloud as a part of this parallelized job:

  rspec:
    executor: rails
    parallelism: 4
    working_directory: ~/my_working_directory
    steps:
      - setup-project
      - run:
          bundle exec rspec --profile 10 \
                            --format RspecJunitFormatter \
                            --out test_results/rspec.xml \
                            --format progress \
                            $(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
      # Save test results for timing analysis
      - store_test_results:
          path: test_results
      - store_artifacts:
          path: coverage
      - sonarcloud/scan

And that is run as part of this workflow:

workflows:
  test:
    when: << pipeline.parameters.run-rails-job >> # This resolves to true in this case
    jobs:
      - rspec:
          context: SonarCloud
      - db
      - rails-lint

The primary issue right now is that each of my 4 test splits get a sonarcloud scan, and their coverage reports are only based on what was run on that particular instance. They end up overwriting each other, so what I really need is a way of merging my coverage results and running only a single SonarCloud scan on the codebase with the merged results.

Anybody know of a solution to this?

2

The CircleCI environment is just running containers that are passed the selected list of tests that are created from the “circleci tests glob” command with the split rule and the parallelism setting being used to decide on how each list the common list is split into unique lists for each CircleCI instance.

If you can not specify within a test the enabling of say a sonarcloud scan the solution is to instead execute a common shell script that takes the test list as a parameter list. This shell script can then perform actions before named tests are executed - such as enabling the sonarcloud scan and directing the output of tests to files or directories that are unique to each test or CircleCI instance.

After the tests have run you can then consolidate the results.

As part of the environment, each of the created CircleCI instances should end up with the environment variable $CIRCLE_NODE_INDEX defined. This is set to the index of the container, with the value being between 1 and n with the maximum possible being the value set in the parallelism line. You can use this to create the unique output files or directories.