# Running Tests **URL:** https://discuss.circleci.com/c/environment/running-tests/58.md [Latest](https://discuss.circleci.com/latest.md) · [Categories](https://discuss.circleci.com/categories.md) · [Tags](https://discuss.circleci.com/tags.md) --- ## [About the Running Tests category](https://discuss.circleci.com/t/about-the-running-tests-category/22984)
**Author:** [@circleci-community](https://discuss.circleci.com/u/circleci-community)\ **Replies:** 0\ **Last updated:** [June 15, 2018, 11:03am UTC](https://discuss.circleci.com/t/about-the-running-tests-category/22984 "2018-06-15T11:03:29Z")
Optimising tests, testing strategies, browser testing, test summary, JUNIT reporting and Insights --- ## [Tests Hanging on Xcode 26](https://discuss.circleci.com/t/tests-hanging-on-xcode-26/54156)
**Author:** [@revans](https://discuss.circleci.com/u/revans)\ **Replies:** 8\ **Last updated:** [September 18, 2026, 8:42pm UTC](https://discuss.circleci.com/t/tests-hanging-on-xcode-26/54156 "2026-09-18T20:42:02Z")
Goal: We want to upgrade CI from Xcode 16.4 → 26.0.0. Along with this we are moving all of our snapshot testing over to iOS 26, on an iPhone 17 Pro. I have my branch made with all the changes, but when I run our testing … --- ## [\[Product Launch\] Smarter Testing is now in Beta](https://discuss.circleci.com/t/product-launch-smarter-testing-is-now-in-beta/54497)
**Author:** [@mseong6251](https://discuss.circleci.com/u/mseong6251)\ **Replies:** 33\ **Last updated:** [August 30, 2026, 11:33pm UTC](https://discuss.circleci.com/t/product-launch-smarter-testing-is-now-in-beta/54497 "2026-08-30T23:33:12Z")
Smarter Testing is now in Beta We’re excited to announce that Smarter Testing is now available in beta for all Cloud customers. This includes test impact analysis, dynamic test splitting, and automatic rerun of failed t… --- ## [Playwright tests show up as failed on Github while running then turn to success when done](https://discuss.circleci.com/t/playwright-tests-show-up-as-failed-on-github-while-running-then-turn-to-success-when-done/54599)
**Author:** [@jack-pap](https://discuss.circleci.com/u/jack-pap)\ **Replies:** 0\ **Last updated:** [May 23, 2026, 10:39am UTC](https://discuss.circleci.com/t/playwright-tests-show-up-as-failed-on-github-while-running-then-turn-to-success-when-done/54599 "2026-05-23T10:39:00Z")
So I have this weird thing happening where my Playwright test suite runs and while in progress it will show up as failed when no tests have failed and at the end of the job turn the job status on github into success. I h… --- ## [Https://discuss.circleci.com/t/environment-variable-not-being-updated-or-read/33355?u=milezdeep9](https://discuss.circleci.com/t/https-discuss-circleci-com-t-environment-variable-not-being-updated-or-read-33355-u-milezdeep9/54453)
**Author:** [@Milezdeep9](https://discuss.circleci.com/u/Milezdeep9)\ **Replies:** 0\ **Last updated:** [March 12, 2026, 12:59pm UTC](https://discuss.circleci.com/t/https-discuss-circleci-com-t-environment-variable-not-being-updated-or-read-33355-u-milezdeep9/54453 "2026-03-12T12:59:06Z")
Continuing the discussion from Environment variable not being updated or read: --- ## [Circleci tests run custom delimiter](https://discuss.circleci.com/t/circleci-tests-run-custom-delimiter/53206)
**Author:** [@Schoko19961](https://discuss.circleci.com/u/Schoko19961)\ **Replies:** 1\ **Last updated:** [January 29, 2026, 2:12am UTC](https://discuss.circleci.com/t/circleci-tests-run-custom-delimiter/53206 "2026-01-29T02:12:08Z")
We have a list of testcases which look like this: Auth Feature Session expired › should redirect to auto logout when session expired Auth Feature Session expired › should redirect to auto logout when switching tab and s… --- ## [\[Product Launch\] Rerun Failed Tests (circleci tests run)](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775)
**Author:** [@sebastian-lerner](https://discuss.circleci.com/u/sebastian-lerner)\ **Replies:** 204\ **Last updated:** [January 14, 2026, 1:47am UTC](https://discuss.circleci.com/t/product-launch-rerun-failed-tests-circleci-tests-run/47775 "2026-01-14T01:47:30Z")
Now GA: Docs: Re-run failed tests - CircleCI. New functionality to rerun failed tests. When selecting this option (see image below), only a subset of tests are rerun, instead of rerunning the entire test suite when a t… --- ## [Jest cannot find setupFilesAfterEnv in circleci](https://discuss.circleci.com/t/jest-cannot-find-setupfilesafterenv-in-circleci/54148)
**Author:** [@muyaszed](https://discuss.circleci.com/u/muyaszed)\ **Replies:** 0\ **Last updated:** [October 9, 2025, 9:43am UTC](https://discuss.circleci.com/t/jest-cannot-find-setupfilesafterenv-in-circleci/54148 "2025-10-09T09:43:08Z")
Hi everyone, I’ve tried searching for solution on this but can’t find anything in the forum. So I have a test work flow using jest which will load a file called /setupTests.js before running the test. Somehow it cannot … --- ## [Run tests that connect to azure key vault to execute tasks](https://discuss.circleci.com/t/run-tests-that-connect-to-azure-key-vault-to-execute-tasks/54086)
**Author:** [@itsuks](https://discuss.circleci.com/u/itsuks)\ **Replies:** 1\ **Last updated:** [September 23, 2025, 4:31pm UTC](https://discuss.circleci.com/t/run-tests-that-connect-to-azure-key-vault-to-execute-tasks/54086 "2025-09-23T16:31:54Z")
I have a microservice application “app1” that runs and connects to azure key vault. When running a complete CI/CD workflow, it perform junit testing and integration testing tasks. The inetgration test needs to connect wi… --- ## [Restoring workspace was overriding test results from previous runs](https://discuss.circleci.com/t/restoring-workspace-was-overriding-test-results-from-previous-runs/54064)
**Author:** [@KieranP](https://discuss.circleci.com/u/KieranP)\ **Replies:** 0\ **Last updated:** [September 8, 2025, 9:03pm UTC](https://discuss.circleci.com/t/restoring-workspace-was-overriding-test-results-from-previous-runs/54064 "2025-09-08T21:03:18Z")
Posting this here incase others experience the same issue, as I couldn’t find an existing issue about this or a support article. I noticed recently that none of our tests (rspec) were being split correctly by previous t… --- ## [Using a checksum to identify a unique directory](https://discuss.circleci.com/t/using-a-checksum-to-identify-a-unique-directory/33737)
**Author:** [@abe](https://discuss.circleci.com/u/abe)\ **Replies:** 3\ **Last updated:** [August 29, 2025, 1:06am UTC](https://discuss.circleci.com/t/using-a-checksum-to-identify-a-unique-directory/33737 "2025-08-29T01:06:39Z")
I have a rails app that compiles assets as part of the test suite. What was working before is no longer working: steps: - checkout - run: mkdir -p tmp/test/cache/assets - attach\_workspace: at… --- ## [JUnit not showing color well in test results](https://discuss.circleci.com/t/junit-not-showing-color-well-in-test-results/50215)
**Author:** [@AmirTugi](https://discuss.circleci.com/u/AmirTugi)\ **Replies:** 1\ **Last updated:** [August 26, 2025, 10:24pm UTC](https://discuss.circleci.com/t/junit-not-showing-color-well-in-test-results/50215 "2025-08-26T22:24:38Z")
I’m running Vitest with a JUnit reporter, and the ansi colors are not shown well in the “tests” tab, while they are shown well in the “steps” tab (using a regular console reporter). This is how it looks like in the test… --- ## [Show screenshot upon failure in 'Test' tab](https://discuss.circleci.com/t/show-screenshot-upon-failure-in-test-tab/53892)
**Author:** [@wjgerritsen-0001](https://discuss.circleci.com/u/wjgerritsen-0001)\ **Replies:** 0\ **Last updated:** [August 4, 2025, 5:07pm UTC](https://discuss.circleci.com/t/show-screenshot-upon-failure-in-test-tab/53892 "2025-08-04T17:07:58Z")
I have a typescript project running a server, and browser tests (using wdio) against the server. I know about the current practice of storing screenshots as artifacts, which is pretty useful. The improvement that I pro… --- ## [Getting net::ERR\_CONNECTION\_REFUSED at http://172.19.0.3:3001/ error when trying to run tests against localhost](https://discuss.circleci.com/t/getting-net-err-connection-refused-at-http-172-19-0-3-3001-error-when-trying-to-run-tests-against-localhost/45314)
**Author:** [@alexkyrylenkoproven](https://discuss.circleci.com/u/alexkyrylenkoproven)\ **Replies:** 3\ **Last updated:** [July 3, 2025, 6:32am UTC](https://discuss.circleci.com/t/getting-net-err-connection-refused-at-http-172-19-0-3-3001-error-when-trying-to-run-tests-against-localhost/45314 "2025-07-03T06:32:52Z")
HI all, I am getting this error when running my tests against the local host-net::ERR\_CONNECTION\_REFUSED at http://172.19.0.3:3001/ error when trying to run tests against localhost I can see that the server is started b… --- ## [Connecting to localhost (localhost)|127.0.0.1|:8080... failed: Connection refused](https://discuss.circleci.com/t/connecting-to-localhost-localhost-127-0-0-1-8080-failed-connection-refused/41335)
**Author:** [@LayMui](https://discuss.circleci.com/u/LayMui)\ **Replies:** 2\ **Last updated:** [July 3, 2025, 6:27am UTC](https://discuss.circleci.com/t/connecting-to-localhost-localhost-127-0-0-1-8080-failed-connection-refused/41335 "2025-07-03T06:27:17Z")
when I run locally, it’s working fine but on the circleci, I can start the web server running here is my config.yml Java Maven CircleCI 2.0 configuration file Check https://circleci.com/docs/2.0/language-java/ for mo… --- ## [Example of splitting by test name for rspec rails](https://discuss.circleci.com/t/example-of-splitting-by-test-name-for-rspec-rails/51459)
**Author:** [@zmf-framework](https://discuss.circleci.com/u/zmf-framework)\ **Replies:** 12\ **Last updated:** [June 17, 2025, 11:09pm UTC](https://discuss.circleci.com/t/example-of-splitting-by-test-name-for-rspec-rails/51459 "2025-06-17T23:09:21Z")
Does anyone here have an example app using --split-by=timings --timings-type=name successfully with Rspec/Rails in a parallelized (20 - 30 containers) set up? I can’t seem to get this to work. My current command is cir… --- ## [Test split by timing-data does not find](https://discuss.circleci.com/t/test-split-by-timing-data-does-not-find/53375)
**Author:** [@lsgndln](https://discuss.circleci.com/u/lsgndln)\ **Replies:** 5\ **Last updated:** [June 12, 2025, 5:30pm UTC](https://discuss.circleci.com/t/test-split-by-timing-data-does-not-find/53375 "2025-06-12T17:30:09Z")
Hey! We recently tried the test splitting technique, and wanted to split them by timing-data. This does not seem to work with our setup. I’ll try to provide as much info as possible below. Test split works fine if we u… --- ## ['Downloading previous test results' step never appears and leads to always missing timings](https://discuss.circleci.com/t/downloading-previous-test-results-step-never-appears-and-leads-to-always-missing-timings/49266)
**Author:** [@sergey-kruk-cobalt](https://discuss.circleci.com/u/sergey-kruk-cobalt)\ **Replies:** 5\ **Last updated:** [June 10, 2025, 9:42am UTC](https://discuss.circleci.com/t/downloading-previous-test-results-step-never-appears-and-leads-to-always-missing-timings/49266 "2025-06-10T09:42:48Z")
We are having trouble splitting tests by timing information. It appears to always complain about missing timings file. And that is understandable as the ‘Downloading tests by timings’ step never appears in the workflow. … --- ## [Pipeline parameter overrides do not appear in UI](https://discuss.circleci.com/t/pipeline-parameter-overrides-do-not-appear-in-ui/53330)
**Author:** [@adriansuarez](https://discuss.circleci.com/u/adriansuarez)\ **Replies:** 4\ **Last updated:** [June 2, 2025, 6:59pm UTC](https://discuss.circleci.com/t/pipeline-parameter-overrides-do-not-appear-in-ui/53330 "2025-06-02T18:59:04Z")
The UI used to show pipeline parameters that were overridden for a pipeline run. Now it does not. Runs with special parameters are indistinguishable from normal runs. --- ## [No timing data found when splitting by testname, and it does outputs file names instead of expected test names](https://discuss.circleci.com/t/no-timing-data-found-when-splitting-by-testname-and-it-does-outputs-file-names-instead-of-expected-test-names/53259)
**Author:** [@henrahmagix](https://discuss.circleci.com/u/henrahmagix)\ **Replies:** 1\ **Last updated:** [May 20, 2025, 2:48pm UTC](https://discuss.circleci.com/t/no-timing-data-found-when-splitting-by-testname-and-it-does-outputs-file-names-instead-of-expected-test-names/53259 "2025-05-20T14:48:35Z")
How can I get circleci tests split --split-by=timings --timings-type=testname to output test names instead of filenames? And correctly find timings data? I’d like to end up with a computed command like: rspec --example… --- ## [Test succeeds locally via SSH tunnel but fails in CircleCI with "url not specified" error](https://discuss.circleci.com/t/test-succeeds-locally-via-ssh-tunnel-but-fails-in-circleci-with-url-not-specified-error/53104)
**Author:** [@hiromaki58](https://discuss.circleci.com/u/hiromaki58)\ **Replies:** 0\ **Last updated:** [April 15, 2025, 4:27am UTC](https://discuss.circleci.com/t/test-succeeds-locally-via-ssh-tunnel-but-fails-in-circleci-with-url-not-specified-error/53104 "2025-04-15T04:27:11Z")
Hello CircleCI Community, I’m facing a problem where my Spring Boot test (PlayerControllerTest.java) runs successfully locally using SSH port forwarding, but fails in CircleCI with the following error: Failed to confi… --- ## [Option to Enable SSH by default](https://discuss.circleci.com/t/option-to-enable-ssh-by-default/53049)
**Author:** [@kbruccoleri](https://discuss.circleci.com/u/kbruccoleri)\ **Replies:** 3\ **Last updated:** [April 4, 2025, 3:56pm UTC](https://discuss.circleci.com/t/option-to-enable-ssh-by-default/53049 "2025-04-04T15:56:48Z")
My organization’s default workflow runs a lot of containers of tests. Retrying with SSH can be a costly process, since it spins up 150 of containers that remain idle while I SSH into just one. I want to make a manual SS… --- ## [CircleCI build fail since Maintenance last Friday](https://discuss.circleci.com/t/circleci-build-fail-since-maintenance-last-friday/53021)
**Author:** [@tony-jidsodsai-yolo](https://discuss.circleci.com/u/tony-jidsodsai-yolo)\ **Replies:** 3\ **Last updated:** [March 31, 2025, 10:59pm UTC](https://discuss.circleci.com/t/circleci-build-fail-since-maintenance-last-friday/53021 "2025-03-31T22:59:19Z")
For no reason a build for test has been failed since the maintenance started last Friday. MSBUILD : error MSB4017: The build stopped unexpectedly because of an unexpected logger failure. System.ArgumentOutOfRangeExcept… --- ## [Why are test insights not available for orgs whose lug is circleci/\?](https://discuss.circleci.com/t/why-are-test-insights-not-available-for-orgs-whose-lug-is-circleci-id/52968)
**Author:** [@marcelloromani](https://discuss.circleci.com/u/marcelloromani)\ **Replies:** 1\ **Last updated:** [March 25, 2025, 1:37pm UTC](https://discuss.circleci.com/t/why-are-test-insights-not-available-for-orgs-whose-lug-is-circleci-id/52968 "2025-03-25T13:37:59Z")
I’ve just started using CircleCI. While I’m extremely pleased so far, I am confused by the documentation of the “upload tests” step, which I added as per the recommendation, but wasn’t producing any test insithts in the … --- ## [Re-run Failed Tests Only Working Once](https://discuss.circleci.com/t/re-run-failed-tests-only-working-once/52879)
**Author:** [@awesammcoder](https://discuss.circleci.com/u/awesammcoder)\ **Replies:** 5\ **Last updated:** [March 14, 2025, 6:00pm UTC](https://discuss.circleci.com/t/re-run-failed-tests-only-working-once/52879 "2025-03-14T18:00:38Z")
I’ve been testing the rerun failed tests feature and noticed some unexpected behavior. On the first test run, if there are failures, I can successfully rerun only the failed tests. However, if a test fails again in the… --- ## ["No timing found for" the same few test files on every new branch](https://discuss.circleci.com/t/no-timing-found-for-the-same-few-test-files-on-every-new-branch/52886)
**Author:** [@henrahmagix](https://discuss.circleci.com/u/henrahmagix)\ **Replies:** 2\ **Last updated:** [March 14, 2025, 12:37pm UTC](https://discuss.circleci.com/t/no-timing-found-for-the-same-few-test-files-on-every-new-branch/52886 "2025-03-14T12:37:37Z")
Every build on a new branch is missing timings data for the same few files, so all our first runs for new feature branches are slow :cry: We see “No timing found” for several files, the same every time on new branch bui… --- ## [How can I add more context to my Slack Message?](https://discuss.circleci.com/t/how-can-i-add-more-context-to-my-slack-message/52877)
**Author:** [@gregory-olshansky\_sa](https://discuss.circleci.com/u/gregory-olshansky_sa)\ **Replies:** 1\ **Last updated:** [March 13, 2025, 12:49pm UTC](https://discuss.circleci.com/t/how-can-i-add-more-context-to-my-slack-message/52877 "2025-03-13T12:49:53Z")
version: 2.1 orbs: katalon-studio: katalon/katalon-studio@23.0.11 slack: circleci/slack@4.12.0 # Include Slack Orb workflows: build: jobs: - katalon-studio/run: version: "latest" co… --- ## [Pytest command not found](https://discuss.circleci.com/t/pytest-command-not-found/52584)
**Author:** [@cyclops45](https://discuss.circleci.com/u/cyclops45)\ **Replies:** 2\ **Last updated:** [January 9, 2025, 12:00am UTC](https://discuss.circleci.com/t/pytest-command-not-found/52584 "2025-01-09T00:00:15Z")
Hi everyone, I have two jobs – “build” and “test”. I would like to use the pip cache from my “build” job to run tests during my “test” job. My pipeline reports that pytest was not found during “test” though it gets inst… --- ## [Is there a way to increase CPU allocation other than increasing resource class?](https://discuss.circleci.com/t/is-there-a-way-to-increase-cpu-allocation-other-than-increasing-resource-class/52445)
**Author:** [@nicholaspshaw](https://discuss.circleci.com/u/nicholaspshaw)\ **Replies:** 3\ **Last updated:** [December 10, 2024, 9:18pm UTC](https://discuss.circleci.com/t/is-there-a-way-to-increase-cpu-allocation-other-than-increasing-resource-class/52445 "2024-12-10T21:18:12Z")
Hello, my team are hitting an issue where test failures are occurring due to 100% CPU utilisation. The easy solution is to increase our resource class on the job to xlarge. Though that increases credits per minute fro… --- ## [CircleCI failed to start the task agent: exec: no command](https://discuss.circleci.com/t/circleci-failed-to-start-the-task-agent-exec-no-command/52406)
**Author:** [@s0unt](https://discuss.circleci.com/u/s0unt)\ **Replies:** 1\ **Last updated:** [December 4, 2024, 9:08pm UTC](https://discuss.circleci.com/t/circleci-failed-to-start-the-task-agent-exec-no-command/52406 "2024-12-04T21:08:59Z")
Hi everyone. I would be really grateful if someone help me with this issue: CircleCI failed to run this build, check your config. Try re-running the build and if this issue persists, open a Support ticket. Detail: coul… [Next page](https://discuss.circleci.com/c/environment/running-tests/58.md?page=1)