I’ve set up my CircleCI configuration to allow retrying failed tests using the following configuration:
- run:
name: Test splitting & run!
command: |
cd src/test/java
# Get list of classnames of tests that should run on this node, excluding classnames of already succeeded.
circleci tests glob "io/channel/api/**/*Test.java" \
| cut -c 1- | sed 's@/@.@g' \
| sed 's/.\{5\}$//' \
| circleci tests run --command=">classnames.txt xargs echo" --split-by=timings --timings-type=classname
# if this is a re-run and it is a parallel run that does not have tests to run, halt execution of this parallel run
[ -s classnames.txt ] || circleci-agent step halt
cd ../../..
GRADLE_ARGS=$(cat src/test/java/classnames.txt | awk '{for (i=1; i<=NF; i++) print "--tests",$i}')
echo "Prepared arguments for Gradle: $GRADLE_ARGS"
./gradlew test -i -PmaxHeapSize=5G $GRADLE_ARGS
Most of the time, when I click “Rerun failed tests”, it correctly retries only the failed tests. However, occasionally, I encounter the following error:
Installing circleci-tests-plugin-cli plugin.
circleci-tests-plugin-cli plugin Installed. Version: 1.0.7332-dfcf8a1
INFO[2024-07-27T01:02:46Z] received failed tests from workflow e729197a-c8d2-4304-a7b9-aa9f1a56b249
INFO[2024-07-27T01:02:46Z] 4 test(s) failed out of 5148 total tests. Rerunning tests from 1 classname
cat: src/test/java/classnames.txt: No such file or directory
Exited with code exit status 1