[Product Launch] Rerun Failed Tests (circleci tests run)

Build Environment Running Tests
63

I was mistakenly passing the file paths twice, which caused the shell to try to execute test files as commands. I’ll post my incorrect config line in case anyone else runs into this issue.
echo $TESTFILES | circleci tests run --command="poetry run python manage.py test $TESTFILES --keepdb" --verbose
with the issue being the second $TESTFILES

1 Like
64

I like https://globster.xyz/ as a tool to help craft an appropriate glob command, I think that should help? Keep in mind you can pass in multiple glob patterns as shown in the snippet here Use the CircleCI CLI to split tests - CircleCI.

65

I am seeing an issue because each projects has its own options and specifies a glob list of files. With circleci tests run handling the file list, they are not associated with their respective project anymore and passing --selectProjects ... grabs all the test files related to the projects specified in jest.config.js.

66

Let me see if replacing it with --projects ... would work. Thanks for the suggestion!

67

Wondering if something like this would work:

$JEST --listTests --selectProjects <project-list> | circleci tests run --verbose --command="xargs -I % $JEST % --selectProjects <project-list>"

The --selectProjects option must appear at the end of the Jest cmd line to work.

68

@gillesbouvier-qz did that end up working for you? My concern would be that jest will always end up using the project-list and overwriting whatever is passed via stdin? For reference, circleci tests run will pass in via stdin to what’s in the --command parameter. If it is a re-run, CircleCI will get the list of failed test file names from the previous job and pass that in via stdin to circleci tests run so that JEST only runs those file names.

69

I realize this feature is still in “preview” but if I see the option greyed out in my “Rerun” dropdown can I expect that this feature is available to me and I just haven’t configured it correctly?

71

Hi @sebastian-lerner, it is a bit more complicated than this, as the --selectedProjects argument needs to always be last.

Here is what I have so far:

  "$JEST" --useStderr --listTests --selectProjects ${JEST_PROJECTS} 2> /dev/null | xargs realpath --relative-to="." |
    circleci tests run --verbose --command="xargs bash -c \"$JEST \$* --selectProjects ${JEST_PROJECTS}\""

Trying it out now. I’m not sure what to do with the coverage report, as it is only going to report for the failed tests. Do you have any guidance on that part?

72

Hmm it’s not working yet. It is re-running all the tests even though it recognized only one needed to be run. By the way, xargs is not the most friendly utility.

73

The dropdown option becomes clickable if and only if two prerequisites are met:

  1. The job has successfully uploaded test results to CircleCI

  2. circleci tests run was used

If both of those are true, then the dropdown is clickable. Are you seeing a case where you meet the prerequisites but the option is not available?

74

@gillesbouvier-qz Fair feedback about xargs. any chance you can send me a job link so I can take a closer look? I wanna see the verbose output. You can send to sebastian @ circleci.com

75

Ah, derp, I think I just needed to refresh the page after the job finished. Seeing it available now. Thanks!

1 Like
77

Hi @sebastian-lerner. Thank you very much for this feature, I believe it would be really useful.

In my case, a shell script is invoked to run my tests, so I have to put circleci tests run in this shell script.

Presumably this will mean I’ll have to download and install a plugin in order for the command to work, as circleci is not recognised in my shell script.

If so, which plugin and where do I download it from?

I plan to use something like curl to retrieve it

78

Hey @orenshafir , could you share the full working step for running the scala tests?

79

Hi @sebastian-lerner , the dropdown is not clickable if I am running circleci tests run from a shell script. This works fine outside of shell script but I can’t figure what why its not working from the shell script.

80

Hey @cristiatob. How did you manage to use the circleci command in the shell script. Did you download and install it somehow? In my case the circleci command is not recognised in my shell script

81

I just used the model from documentation , I did not install anything .

82

Thanks @cristiatob.

I understand that when you use circleci tests run in config.yml it works because the circleci command is recognised there.

But once your outside of the config.yml e.g. in a shell script, it has not been possible for me to simply call circleci tests run as the shell script has no understanding of what the circleci command is. Which leads me to think that some sort of curl request is required to download the appropriate binary that can be used to execute that command from within the shell script.

Could you kindly point me to the link to the documentation that has the model you were referring to?

83

Larry, I generally recommend using this method if you are running tests from within a shell script: https://circleci.com/docs/rerun-failed-tests-only/#output-test-files-only

This will let you from your config.yml get the list of test filenames that are going to be run (either on the original job run or the re-run). Then you can pass those filenames into your shell script to invoke with your test command. circleci tests run is not technically needed to actually execute the tests. Let me know if something like this is feasible for you.

84

@cristiatob Can you send me (sebastian @ circleci.com) a link to a job where you’re seeing this? Whether circleci tests run is invoked from within a shell script or not should not have an impact on the button being clickable…