Pip install step failing

Build Environment
1

We’re on Circle 2.0. In a step where Python dependencies are being installed, something like:

- run:
      name: install python dependencies
      command: |
        python -m virtualenv venv
        . venv/bin/activate
        pip install -r requirements/internal_modules_dev.txt --no-cache-dir
        pip install -r requirements/requirements.txt

the step fails with an exit code 1.
Exited with code 1

This started happening 4 days ago, and all our builds after April 15, 2018 3:00 AM UTC have been failing.

When rebuilding with SSH and performing the same step manually, though, everything works.

3

Apologies. Done.

1 Like
4

Which of these commands is failing? I can see four here. That might help narrow it down.

We might need to see the whole of your config file. Do you have other containers/images that you are relying on, for example? Containers spin up in parallel, so you can get a race condition unless you wait for required services explicitly.

Bear in mind that mostly problems of this kind are not Circle specific problems, and they just require some patience and persistence to solve. For example, if you put sleep 10 on a separate line before the failing command, does that help? If it does, find out why.

5

I’m having similar issues with a pip install step. We are still using version 1. The first time I saw the issue was on Apri 16th so the timescales seem to line up.

The specific error I’m getting is

AssertionError: No source dir for django-model-utils

My requirements.txt file hasn’t changed and neither has my circle config. To date, a failed build has always worked a second time if I choose to ‘Rebuild without cache’, which makes me think this is something to do with a change to how the cache works.

We have ~50 odd modules to install in the requirements file. I’ve seen the issue with two different modules. Both the failing modules are when we’ve had to install a specific commit from GitHub, rather than using pypi.

e.g:

git+git://github.com/jazzband/django-model-utils.git@343d45940616b92836011a5257d57e4136a72296#egg=django-model-utils

FWIW: We automatically upgrade pip as part of the pre build

- pip install --upgrade pip
- pip install -r requirements.txt
6

May we see how you are using the cache, in your config file? Does removing the cache directives (and doing all 50 dep fetches every time) solve the problem? It’d be slower, but it might be worth running like that for a few weeks to see if that resolve it (and thus points to a cache problem).

7

This is my config file. We don’t (afaik) have anything specific to the cache.

The circle config and the requirements.txt haven’t been changed in 2 months, and have been working up until April 16th hence my belief in thinking something else must have changed.

(Its only in copying into a public forum I’ve noticed the nonsensical else statement in the requirements.txt statement. I’ve left in for the sake of being honest at the risk of my own embarrassment!)

machine:
  python:
    version: 3.4.3
  node:
    version: 8.1.4
  environment:
    PATH: "${PATH}:${HOME}/${CIRCLE_PROJECT_REPONAME}/front-end/node_modules/.bin"
dependencies:
  pre:
    - pip install --upgrade pip
    - pip install --upgrade 
    - if [ -e requirements.txt ]; then pip install -r requirements.txt;else pip install -r requirements.pip;fi
  override:
    - yarn:
        pwd: front-end
test:
  override:
    - coverage run manage.py test --settings 
    - yarn test:
        pwd: front-end
  post:
    - coverage html -d $CIRCLE_ARTIFACTS
8

I am not an expert on Circle internals, but I don’t think the “Rebuild without cache” will affect you, Duncan. I suspect you have a flaky built or race condition and it just works by coincidence on the next try.

Cache statements (which can be used or ignored) look like this:

  - restore_cache:
      keys:
        - v1-{{ .Branch }}
      paths:
        - /caches/my-image.tar

  # Do cacheable stuff in a 'run' step here

  - save_cache:
      key: v1-{{ .Branch }}
      paths:
        - /caches/my-image.tar
9

Might be worth fixing that now, so the better/simpler version can bed in?

10

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.