Unable to save test results from Cypress test

[lritter79]

Goal: I want to be able to store my Cypress test results so I can start using the re-run failed tests option in CircleCI.
Problem. On the “Uploading Test Results” step of my Cypress tests, I get the following error

Unable to save test results from /root/project/cypress
Error error accessing path "/root/project/cypress": stat /root/project/cypress: no such file or directory
Found no test results, skipping

I’m using the orb cypress-io/cypress@2.2.0 to run Cypress.
Here’s the relevant code from my circleci config:

workflows:
  main:
    jobs:
    - cypress/run:
              name: "Cypress Tests"
              executor: cypress-18-16-1
              no-workspace: true
              context:
                - "AWS CLI"
                - "Cypress"
              cache-key: dependencies-{{ checksum "frontend/yarn.lock" }}
              parallelism: 10
              timeout: 20m
              post-checkout:
                - run: apt-get update && apt-get install curl -y
                - run: apt-get update && apt-get install unzip -y
                - run: echo 'export CODEARTIFACT_AUTH_TOKEN=`aws codeartifact get-authorization-token --domain ottertune --domain-owner 691523222388 --query authorizationToken --output text --profile default`' >> "$BASH_ENV"
                - ot/codeartifact-get-auth-token
              command: |
                TZ=America/Los_Angeles yarn cypress run --env split=true --reporter cypress-circleci-reporter
              post-steps: 
                - store_test_results: 
                    path: cypress
              working_directory: /root/project/frontend
              yarn: true
              start: yarn start
              store_artifacts: true

[sebastian-lerner]

could it be saving them in /root/project/frontend/cypress?

also did you add the cypress-circleci-reporter to your package.json? I think that reporter will save things in a test_results directory by default? cypress-circleci-reporter - npm

[lritter79]

Yes, I did add cypress-circleci-reporter to my package.json file.

I don’t think it’s saving them anywhere, nothing shows up under the “tests” tab on the Cypress job after ti runs

[sebastian-lerner]

it won’t show up in the tests tab unless the store_test_results step that you put in your first message has a successful upload.

so that’s why i’m wondering if maybe cypress is storing them in a test_results directory and then if you change store_test_results to look at test_results instead of the cypress path, maybe they are there?

[rit1010]

A simple question - what is your executor cypress-18-16-1 defined as?

Without knowing what environment you have defined I can not give a clear answer, but in most environments, the job would not be running as the root user and so would not have access to the root directory.

The easy way to check would be to just place the following into the script

            pwd
            whoami
            echo hi > /root/testfile

As that would provide the current working directory, the account that is being used to run the job and a basic check to see if you can write into the root directory.

[lritter79]

@lritter79

I tried keeping the rest of the config the same and changing the post-steps to this:

            - store_test_results: 
                path: test_results

and I got this error:

Unable to save test results from /root/project/test_results
Error error accessing path "/root/project/test_results": stat /root/project/test_results: no such file or directory
Found no test results, skipping```

Is that what you meant? If not, what should I change?

[lritter79]

Thanks for the insight, but can you be more specific about where to run this code in the my circleci config?

[rit1010]

after your post-checkout: add the following (with the correct padding)

         command: |
            pwd
            whoami
            echo hi > /root/file

much like you have your current command: line

when I add it to a test job my output looks like this

/home/circleci/project
circleci
/bin/bash: line 3: /root/file: Permission denied

Which indicates that my job is being run under the circleci user, which does not have write access to /root

[lritter79]

This is what I get on my output:

/root/project/frontend
root

[rit1010]

OK, so the user that is not the issue.

The next question I have to ask is why are you using such a dated version of the cypress ORB? 2.2.0 is from Sept 2022, while you are defining a very recent version of Node to be the executor environment.

As Sebastian noted in one of his replies this seems to be an issue with directories. If the working directory is set as /root/project/frontend the cypress directory should end up as /root/project/frontend/cypress

Beyond those comments/ideas you may want to see what level of support the cypress team can give users of the ORB as it is their code base rather than CircleCI’s. They have a community chat on Discord at https://discord.gg/cypress

[lritter79]

We are on version 2.2.0 because the migration process to version 3.3.0 was not straightforward, and that we also use version 12 of Cypress, which was the latest major version at the time we set up our tests. There is a /root/project/frontend/cypress directory, it where the videos from the tests are saved. When it comes to storing test results, I just doesnt work.

[rit1010]

I think you may be able to get more support from the Cypress discord forum as there should be a wider range of active users there.

With the answers you have provided, the issue does seem to be within the configuration of the Cypress ORB rather than the CircleCI environment, but without decoding the source code of the ORB it is hard to tell exactly what is going wrong.

[sebastian-lerner]

Can you add a step that is:

ls /root/project/frontend/cypress

and another step that does

ls/root/project/

I just want to see the directories in there to rule out that the test results aren’t actually getting generated and we’re just missing the right path

[lritter79]

Sure, here’s what gets printed on running ls /root/project/frontend/cypress:

e2e  fixtures  full-service  plugins  support

These are the folders inside the cypress folder in our front end React React project we keep our tests and stuff in.

Here’s the output of ls/root/project/:

Makefile  README.md  frontend

[sebastian-lerner]

hmm…okay

what happens if you just try to use a junit reporter instead of the circleci reporter? by following the example here: GitHub - cypress-io/cypress-example-docker-circle: Cypress + Docker + CircleCI = ❤️

i’m trying to pinpoint if it’s the reporter that’s the issue or the cypress invokation. i’m also wondering if the old cypress orb might be an issue as well…

[lritter79]

I changed it to use the reporter from the example, and it still didnt save the test results

[sebastian-lerner]

I feel like it might be the old cypress orb then. can you send me a link to a job i want to look a little more closely sebastian@circleci.com

[lanice]

Hi! Sorry for the delay, I’m picking this back up from @lritter79. We did decide to upgrade the Cypress orb after all, since you’re right, the version we were using was quite old. Now we are on version 3.3.0, and our tests are running again. I do get a permissions error now during the “Restore Cypress cache” step that looks like this (but ca 25k lines):

Unable to set permissions on "/root/.cache/Cypress/12.17.4/Cypress/resources/[...]" - chmod /root/.cache/Cypress/12.17.4/Cypress/resources/app/[...]: permission denied

And now, after the upgrade, the pwd / whoami / echo hi > /root/file command give the following output:

/home/circleci/project/frontend
circleci
/bin/bash: line 3: /root/file: Permission denied

So now the job is run under the circleci user (before it was root). Is there a way for us to give the circleci user the permissions it needs?

[rit1010]

The Cypress ORB allows you to set your own cache-path

CircleCI Developer Hub - cypress-io/cypress

The original posted script also contained

working_directory: /root/project/frontend

So you may have other code that needs to be changed so that you are not dependent on the root account.

If you use the default environment defined by the ORB you can not use the root account as it defines a docker based environment. You could override this to use a machine environment, but it would be best to ask in the Cypress forums as they are the ones who know the code.

[lanice]

Thanks for the pointer towards the cache-path. Unfortunately it does not seem to be respected? This is the part of my current circleci config:

version: 2.1
orbs:
  cypress: cypress-io/cypress@3.3.0
workflows:
  main:
    jobs:
      - cypress/run:
          name: "Cypress Tests"
          context:
            - "AWS CLI"
            - "Cypress"
          cypress-cache-key: dependencies-{{ checksum "frontend/yarn.lock" }}
          cypress-cache-path: ~/frontend/.cache/Cypress
          include-branch-in-node-cache-key: true
          parallelism: 10
          working-directory: frontend
          package-manager: yarn
          install-command: yarn install
          post-install: yarn cypress install
          start-command: yarn start
          cypress-command: TZ=America/Los_Angeles yarn cypress run --env split=true --reporter cypress-circleci-reporter

And the output of the Restore Cypress Cache step is still:

Found a cache from build 3112 at dependencies-mi+Nw7G5384bt6AayAzAUJ8wzdRpkRyGcL9UCAuTZRE=
Size: 174 MiB
Cached paths:
  * /root/.cache

Downloading cache archive...
Validating cache...
Download duration for cache file /tmp/cache2348385590: 4.815458482s

Unarchiving cache...
Skipping writing "root/.cache/" - mkdir /root/.cache: permission denied
Skipping writing "root/.cache/Cypress/" - mkdir /root/.cache: permission denied
[...]

I’ve tried /home/circleci/.cache/Cypress for the cache path as well, same result.