I’m working with a Rails project that has a fairly large set of tests. These tests have been executed on Circle using the ruby orb and the rspec-test task.
When tests get executed using
around 300 tests are executed.
When I run using rspec/bundle directly (not using the ruby orb) using
command: "bundle exec rspec --order random"
name: Run rspec tests
around 2700 tests executed. Any idea what’s going on? I’m using the email@example.com orb but I’ve been able to replicate this with the firstname.lastname@example.org orb.
to confirm, you’re expecting 2700 tests to be executed correct? Are you setting any parallelism level?
If you set the
rerun-fail param to
false, does it make a difference in the number of tests that are run??
This is correct. I’m expecting to see 2700 tests executed.
I’m not setting any parallelism in these runs, using rspec or otherwise.
Interesting, could you try setting the rerun-fail param explicitly to false like I mentioned above and letting me know how many tests get executed?
I used the
2.1.0 version of the Ruby orb and re-ran the tests with the
rerun-fail parameter set to false and there were still 900 tests run instead of 2700.
Got it. Any chance you could send me a link to one of your jobs to email@example.com so I can take a closer look?