Rerun failed tests first on subsequent git commit on PR branch

Build Environment
1

Hi, we use Rails/Cucumber/RSpec, and run CircleCI on pull request branches. Our full test suite takes about 25 minutes to run on CircleCI.

If we have a few failing tests, we then do a subsequent git commit, it would be great if the first set of test that get run are the ones that failed last time. Then, if these fail, it stops, and if they pass, it then goes on to run the full suite.

That way we could avoid waiting 25 minutes only to find the same tests failed again.

Anyone know if CircleCi can do this already?

Thanks!

1 Like
2

This is an interesting question - I’m going to ask around and see if anyone knows how to do this!

3

Hi @thekatertot did you get any ideas on this?

4

Hi @pw60462! Thanks for your patience. I think the best solution is rerun from failed, which should cut down your build time by not repeated the passed tests. Here’s a short article on where to find it in the UI. If it doesn’t work for you, let me know!

5

That reruns an entire failed step. Usually all the specs are in one step, so I suppose people could split it up. I’m not familiar enough to know if that would impair it’s ability to parallelize the specs itself or not. I’m also interested in figuring out how we might be able to do this, because it would save a lot of time and CPU credits.

6

I saw a banner at the top in the web UI which I believe was saying there’s a new feature being tested that, is supposed to be able to do that. There was a Google form for signup.

7

Is what the OP asked possible yet? We have a Cypress test suite that takes ~25 to run. If some tests fail and I fix the cause of the failure and push changes, it’d be great if the tests that failed previously could be run first.

Note, this is not the same as “rerun from failed” which is for re-running the current workflow for flaky tests. I’d like the next run to start by running the failing tests first.

If splitting tests by time is possible then it should be possible to retain information on the failed tests so they can be run first the next time.

8

@mallison there is not a feature to run the failed tests first on the next run. There is a feature that reruns only the failed tests when you have flaky tests. But like you called out, this is on the same run, not after youve made a fix and started a new run

https://circleci.com/docs/rerun-failed-tests-only/