# \[Product Launch\] Rerun Failed Tests (circleci tests run)

**URL:** <https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775>\
**Category:** Running Tests\
**Created:** [April 14, 2023, 3:15pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775 "2023-04-14T15:15:58Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![sebastian-lerner](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sebastian-lerner/32/27833_2.png) [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\
**Post date:** [April 14, 2023, 3:15pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/1 "2023-04-14T15:15:58Z")

</div>

Now GA:

**Docs** : [Re-run failed tests - CircleCI](https://circleci.com/docs/rerun-failed-tests-only/).

New functionality to **rerun failed tests**. When selecting this option (see image below), only a subset of tests are rerun, instead of rerunning the entire test suite when a transient test failure arises.

Historically, when your testing job in a workflow has flaky tests, the only option to get to a successful workflow was to [rerun your workflow from failed](https://support.circleci.com/hc/en-us/articles/360050303671-How-To-Rerun-a-Workflow). This type of rerun executes **all tests** from your testing job, including tests that passed, which prolongs time-to-feedback and consumes credits unneccessarily.

This functionality reruns failed tests from the _same_ commit, not new ones.

 ![image](https://global.discourse-cdn.com/circleci/original/3X/8/2/8276dddb75f8133cb27aec8b2b05d5a8f620dee8.png)

If you have issues setting up and/or using this functionality, comment below.

---

<div class="post-metadata">

**Author:** ![heyogrady](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/heyogrady/32/32924_2.png) [@heyogrady](https://discuss.circleci.com/u/heyogrady)\
**Post date:** [May 2, 2023, 11:36pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/2 "2023-05-02T23:36:25Z")

</div>

This is fantastic feature. Has anyone gotten this to work in a test suite also submits coverage to CodeClimate? Trying to figure out how to get coverage to pass when the workflow run is only running a few of the tests.

---

<div class="post-metadata">

**Author:** ![sebastian-lerner](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sebastian-lerner/32/27833_2.png) [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\
**Post date:** [May 3, 2023, 8:59pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/3 "2023-05-03T20:59:26Z")

</div>

@heyogrady I’m curious what error message you’re getting back from CodeClimate? Is it the lack of a coverage report or an incomplete report?

I’m talking to another user where I’m suggesting adding in some logic which says, “if report exists, pass it along; else, do nothing” (ie. don’t fail)

---

<div class="post-metadata">

**Author:** ![miyucy](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/miyucy/32/27119_2.png) [@miyucy](https://discuss.circleci.com/u/miyucy)\
**Post date:** [May 9, 2023, 1:54am UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/5 "2023-05-09T01:54:25Z")

</div>

I’m having trouble because the following error has occurred.

```auto
Error: Failed to download and install circleci-tests-plugin-cli plugin: Error downloading circleci-tests-plugin-cli: rename /tmp/circleci-tests-plugin-cli/1.0.5743-3632d00/linux/amd64/circleci-tests-plugin-cli.tmp /tmp/circleci-tests-plugin-cli/1.0.5743-3632d00/linux/amd64/circleci-tests-plugin-cli: no such file or directory

```

---

<div class="post-metadata">

**Author:** ![sebastian-lerner](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sebastian-lerner/32/27833_2.png) [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\
**Post date:** [May 9, 2023, 2:15am UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/6 "2023-05-09T02:15:06Z")

</div>

@miyucy can you send a link to the job where you’re seeing that please? Sebastian @ [circleci.com](http://circleci.com)

---

<div class="post-metadata">

**Author:** ![mehtarupesh](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/mehtarupesh/32/32971_2.png) [@mehtarupesh](https://discuss.circleci.com/u/mehtarupesh)\
**Post date:** [May 11, 2023, 4:45am UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/7 "2023-05-11T04:45:31Z")

</div>

Hi, the feature is awesome to have!

I have modified my yml file referring to the python example in the docs. The job runs (and has test failures as expected) but the feature (rerun failed tests only) is not showing up in UI post the run.

Any suggestions on what could be going wrong ? Below is the job info from my config.yml. Thanks !

```auto
version: 2.1

orbs:
  aws-cli: circleci/aws-cli@2.0.6
  coveralls-private: kognitos/coveralls-private@0.0.4

commands:
  test-component:
    docker:
      - image: cimg/python:3.9.8-node
    resource_class: xlarge
    environment:
      DOCKER_SOCKS_PROXY: 127.0.0.1:1080
    parallelism: 12
    steps:
      - checkout
      - add-ssh-keys
      - aws-cli/setup
      - install-ssm-plugin
      - install-psql-client
      - setup_remote_docker:
          version: 20.10.2
          docker_layer_caching: true
      - attach_workspace:
          at: "."
      - run: make configure-docker-ubuntu
      - run:
          name: "setup docker proxy"
          command: ssh -D ${DOCKER_SOCKS_PROXY} -N remote-docker
          background: true
      - run:
          name: test component
          parallel: true
          command: |
            source venv/bin/activate
            mkdir test-component-results
            TEST_FILES=$(circleci tests glob "tests/component/test_*.py")
            echo $TEST_FILES | circleci tests run --command=" \
            xargs pytest \
            -o junit_family=legacy \
            --numprocesses=24 \
            --maxprocesses=24 \
            --dist load \
            --junitxml=test-component-results/junit.xml \
            --cov-report lcov:./coverage/test/cov.info \
            --cov-append \
            --cov=./src \
            --timeout=600 \
            --timeout_method=thread \
            --log-cli-level=ERROR" \
            --split-by=timings
      - store_test_results:
          path: test-component-results
      - coveralls-private/upload:
          parallel: true
          build_num: CIRCLE_PIPELINE_NUM
          flag_name: Brain Tests
          path_to_lcov: ./coverage/brain/cov.info
      - store_artifacts:
          path: test-brain-results

```

---

<div class="post-metadata">

**Author:** ![sebastian-lerner](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sebastian-lerner/32/27833_2.png) [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\
**Post date:** [May 11, 2023, 12:25pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/8 "2023-05-11T12:25:13Z")

</div>

Hi @mehtarupesh, the feature is currently in a _closed_ preview/beta. We’re hoping to open this up to all users in the upcoming weeks at which point you’ll have the option available in the UI.

---

<div class="post-metadata">

**Author:** ![basti1302](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/basti1302/32/32978_2.png) [@basti1302](https://discuss.circleci.com/u/basti1302)\
**Post date:** [May 11, 2023, 12:29pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/9 "2023-05-11T12:29:35Z")

</div>

I am also seeing “Error: Failed to download and install circleci-tests-plugin-cli plugin: Error downloading circleci-tests-plugin-cli: rename /tmp/circleci-tests-plugin-cli/1.0.5743-3632d00/linux/amd64/circleci-tests-plugin-cli.tmp /tmp/circleci-tests-plugin-cli/1.0.5743-3632d00/linux/amd64/circleci-tests-plugin-cli: no such file or directory”

I think it might be happening when you execute “circleci tests run” in parallel more than once on a single executor. I do not encounter the issue when I execute “circleci tests run” only once. As soon as it is exected a second time, the install fails with this error message. So my guess is that “circle tests run” trigger this installation implicitly, but it is not safe to run multiple installations in parallel.

Here is why we would execute the command multiple times. We have a monorepo with multiple npm packages in one repo. Our CircleCI config triggers the tests in all packages via lerna (a Node.js monorepo management tool) in parallel. The test commands in the individual packages then start the actual tests via “circleci tests run”, and as soon as this is started for the second package, the install fails.

Link to the failed job: https --app.circleci.com/pipelines/github/instana/nodejs/5714/workflows/d5d37991-1869-42eb-bf12-4a805f70126c/jobs/20147

---

<div class="post-metadata">

**Author:** ![miyucy](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/miyucy/32/27119_2.png) [@miyucy](https://discuss.circleci.com/u/miyucy)\
**Post date:** [May 11, 2023, 12:53pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/10 "2023-05-11T12:53:02Z")

</div>

Thank you! My problem was exactly the same as yours.  
I resolved this issue by running `circleci tests run --help` before executing the test to complete the installation process.

---

<div class="post-metadata">

**Author:** ![basti1302](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/basti1302/32/32978_2.png) [@basti1302](https://discuss.circleci.com/u/basti1302)\
**Post date:** [May 12, 2023, 7:34am UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/11 "2023-05-12T07:34:58Z")

</div>

Oh, nice! Thanks for sharing, that works like a charm.

---

<div class="post-metadata">

**Author:** ![sebastian-lerner](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sebastian-lerner/32/27833_2.png) [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\
**Post date:** [May 15, 2023, 2:58pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/12 "2023-05-15T14:58:26Z")

</div>

One update, we saw a handful of reports of users running into intermittent jobs that failed with exit code 141. We have just pushed what we believe is a fix to that issue.

If you see a job fail with exit code 141, please let me know!

---

<div class="post-metadata">

**Author:** ![villelahdenvuo](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/villelahdenvuo/32/17891_2.png) [@villelahdenvuo](https://discuss.circleci.com/u/villelahdenvuo)\
**Post date:** [May 16, 2023, 12:40pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/13 "2023-05-16T12:40:58Z")

</div>

Hey, it seems to work nicely, I can re-run my failed Cypress tests, however I see this error in the logs right after Cypress results:

```auto
WARN[2023-05-16T12:10:36Z] There was an error when uploading test results from with batchID 3c53b247-4b13-4b63-ba9d-2f41dca214e3: no path given 
DEBU[2023-05-16T12:10:36Z] Error encountered with test batch 3c53b247-4b13-4b63-ba9d-2f41dca214e3 
INFO[2023-05-16T12:10:36Z] ending execution

```

What is that about?

---

<div class="post-metadata">

**Author:** ![villelahdenvuo](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/villelahdenvuo/32/17891_2.png) [@villelahdenvuo](https://discuss.circleci.com/u/villelahdenvuo)\
**Post date:** [May 16, 2023, 12:55pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/14 "2023-05-16T12:55:08Z")

</div>

Also, I do get this warning from Cypress:

> ⚠ Warning: It looks like you’re passing --spec a space-separated list of arguments: “xxx yyy zzz”
> 
> This will work, but it’s not recommended.
> 
> If you are trying to pass multiple arguments, separate them with commas instead:
> 
> cypress run --spec arg1,arg2,arg3
> 
> The most common cause of this warning is using an unescaped glob pattern. If you are
> 
> trying to pass a glob pattern, escape it using quotes:
> 
> cypress run --spec “\*\*/\*.spec.js”

Maybe it would be nice to have an option e.g. circleci test run --file-delimiter=“,”

---

<div class="post-metadata">

**Author:** ![sebastian-lerner](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sebastian-lerner/32/27833_2.png) [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\
**Post date:** [May 16, 2023, 1:17pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/15 "2023-05-16T13:17:43Z")

</div>

> [@villelahdenvuo](#):
>
> Hey, it seems to work nicely, I can re-run my failed Cypress tests, however I see this error in the logs right after Cypress results:
> 
> ```auto
> 
> ```

Those are warnings and debugging messages that we are in the process of removing to avoid confusion. Sorry about that!

Regarding the delimiter, let me pass that back to the team and see what can we do. Thanks for the feedback!

---

<div class="post-metadata">

**Author:** ![drilon241](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/drilon241/32/32382_2.png) [@drilon241](https://discuss.circleci.com/u/drilon241)\
**Post date:** [May 17, 2023, 3:35pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/16 "2023-05-17T15:35:31Z")

</div>

hey @villelahdenvuo I am trying to get this working for our cypress tests but hitting some issues.  
We currently use the circleci test split command but when trying to change that to use tests run instead as per docs we get errors. `Can't run because no spec files were found.`  
Do you mind sharing an example of your setup?

---

<div class="post-metadata">

**Author:** ![sebastian-lerner](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sebastian-lerner/32/27833_2.png) [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\
**Post date:** [May 17, 2023, 4:02pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/17 "2023-05-17T16:02:48Z")

</div>

@drilon241 if you send over your config.yml to sebastian @ [circleci.com](http://circleci.com) I can take a look at well. Were you following the cypress instructions in the docs [Re-run failed tests only (preview) - CircleCI](https://circleci.com/docs/rerun-failed-tests-only/#configure-a-job-running-cypress-tests)?

---

<div class="post-metadata">

**Author:** ![chadchabot](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/chadchabot/32/32584_2.png) [@chadchabot](https://discuss.circleci.com/u/chadchabot)\
**Post date:** [May 17, 2023, 4:12pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/18 "2023-05-17T16:12:36Z")

</div>

If you’d be wiling to share it, @villelahdenvuo, can you paste the command you’re using with the `circleci tests run` tool? (obscure whatever path or project name information you’d like)

I think you could have the result you’re looking for (the list of values sent to the -spec flag comma separated) by using the `tr` translate tool, like so:

```auto
$ echo "arg1 arg2 arg3" | tr ' ' ',' | xargs -I {} cypress run -spec {}

cypress run -spec arg1,arg2,arg3

```

Removing the initial `echo` command, and having the contents of the `--command` flag be `--command="tr ' ' ',' | xargs -I {} cypress run -spec {}"`.

The tool currently expects input to the tool to be space or newline delimited, but how it’s assembled or arranged for the specific test runner you’re using is very flexible, depending on your comfort and familiarity with built-in bash tools.

---

<div class="post-metadata">

**Author:** ![drilon241](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/drilon241/32/32382_2.png) [@drilon241](https://discuss.circleci.com/u/drilon241)\
**Post date:** [May 18, 2023, 4:12pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/19 "2023-05-18T16:12:35Z")

</div>

Config sent via email. Thanks for looking into this 🙏  
I was following the official docs but maybe I missed something.

---

<div class="post-metadata">

**Author:** ![villelahdenvuo](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/villelahdenvuo/32/17891_2.png) [@villelahdenvuo](https://discuss.circleci.com/u/villelahdenvuo)\
**Post date:** [May 18, 2023, 4:23pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/20 "2023-05-18T16:23:56Z")

</div>

I just followed the example on the circleci docs for cypress and it worked.

---

<div class="post-metadata">

**Author:** ![villelahdenvuo](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/villelahdenvuo/32/17891_2.png) [@villelahdenvuo](https://discuss.circleci.com/u/villelahdenvuo)\
**Post date:** [May 18, 2023, 4:27pm UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775/21 "2023-05-18T16:27:55Z")

</div>

Thanks, I’ll try your way. It currently works with the spaces, just gives a warning.

I personally don’t love the idea of cramming a complex piped command in a string parameter, though. It doesn’t seem very readable for humans.

[Next page](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775.md?page=2)