Pytest command not found

Build Environment Running Tests
1

Hi everyone,

I have two jobs – “build” and “test”. I would like to use the pip cache from my “build” job to run tests during my “test” job. My pipeline reports that pytest was not found during “test” though it gets installed from a setup.py file during “build”. I would greatly appreciate thoughts on how I can fix this issue. Here are the contents of my config.yml file.

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/python:3.12.7
    steps:
      - checkout
      - restore_cache:
          key: &proj-cache proj-{{ .Branch }}-{{ checksum "setup.py" }}
      - run:
          name: Install dependencies
          command: |
            python3 -m venv venv
            . venv/bin/activate
            pip install .
      - save_cache:
          key: *proj-cache
          paths:
            - "venv"

  test:
    docker:
      - image: cimg/python:3.12.7
    steps:
      - checkout
      - restore_cache:
          key: &proj-cache proj-{{ .Branch }}-{{ checksum "setup.py" }}
      - run:
          name: Run tests
          command: |
            . venv/bin/activate
            pytest -v tests/

workflows:
  version: 2
  build_and_test:
    jobs:
      - build
      - test:
          requires:
            - build

Thanks!

2

i used the “ask AI” button on the right hand side of this page and i think it is giving the right answer? let me know if this doesnt solve it though

Based on the information provided in the knowledge sources and your config.yml file, I can see why you’re experiencing this issue. Let me explain the problem and suggest a solution.
The main issue is that while you’re saving and restoring the cache correctly, you’re not persisting the virtual environment between jobs. Each job in CircleCI runs in a separate container, so the venv directory created in the build job is not automatically available in the test job, even if you restore the cache.
Here’s how you can fix this:
Use persist_to_workspace in the build job to save the virtual environment.
Use attach_workspace in the test job to make the virtual environment available.
Here’s the modified config.yml that should work:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/python:3.12.7
    steps:
      - checkout
      - restore_cache:
          key: &proj-cache proj-{{ .Branch }}-{{ checksum "setup.py" }}
      - run:
          name: Install dependencies
          command: |
            python3 -m venv venv
            . venv/bin/activate
            pip install .
      - save_cache:
          key: *proj-cache
          paths:
            - "venv"
      - persist_to_workspace:
          root: .
          paths:
            - venv

  test:
    docker:
      - image: cimg/python:3.12.7
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run:
          name: Run tests
          command: |
            . venv/bin/activate
            pytest -v tests/

workflows:
  version: 2
  build_and_test:
    jobs:
      - build
      - test:
          requires:
            - build

The key changes are:
In the build job, we’ve added a persist_to_workspace step to save the venv directory.
In the test job, we’ve replaced the restore_cache step with an attach_workspace step to make the venv directory available.
These changes ensure that the virtual environment created and populated in the build job is available in the test job, including all installed packages like pytest.
For more information on using workspaces to share data between jobs, you can refer to the CircleCI documentation on Persisting Data Between Jobs.
This solution should resolve your issue with pytest not being found in the test job.

1 Like
3

Just my personal $0.02 (which you can also verify in Circle’s docs) – if you want to use Python caching with CircleCI, using a tool like Poetry or Pipenv will tend to simplify things a bit, and should allow you to more easily do partial cache restore, and should allow you to do it without using persist_to_workspace, I think.

I would use poetry over pipenv, personally; the approach should be similar either way. Could also look at either directly using (or using as an example) some of the stuff in the python orb, which supports both IIRC.

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.