# Python pip dependencies are not cached, reinstalled every time

**URL:** <https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965>\
**Category:** Bug Reports\
**Tags:** pip, cache, python\
**Created:** [September 23, 2016, 2:31pm UTC](https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965 "2016-09-23T14:31:19Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ego](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ego/32/31011_2.png) [@ego](https://discuss.circleci.com/u/ego)\
**Post date:** [September 23, 2016, 2:31pm UTC](https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965/1 "2016-09-23T14:31:19Z")

</div>

When I run our build script locally I see lines like:

```auto
Collecting requests==2.4.3 (from contentful.py==0.9.3->-r requirements.txt (line 4))
  Using cached requests-2.4.3-py2.py3-none-any.whl

```

With no ‘[setup.py](http://setup.py) install’ step for the majority of packages (the ones which come from pypi rather than github)

However on CircleCI it downloads the package every time, for each one:

```auto
Collecting requests==2.4.3 (from contentful.py==0.9.3->-r requirements.txt (line 4))
  Downloading requests-2.4.3-py2.py3-none-any.whl (459kB)
    16% |█████▏ | 73kB 23.0MB/s eta 0:00:01 100% |████████████████████████████████| 462kB 1.2MB/s 

...

  Running setup.py bdist_wheel for requests-oauthlib ... - done
  Stored in directory: /home/ubuntu/.cache/pip/wheels/73/f4/27/df0ae9b4e83413bd9bb24d42a4286eb77a227661c0e2de8fc2

```

Sadly we are using a bunch of packages like Pandas, numpy etc which take FOREVER to install

I have tried adding:

```auto
dependencies:
  cache_directories:
    - /home/ubuntu/.cache/pip

```

but it makes no difference

We’re currently evaluating CircleCI at my company and it’s unusable unless this can be resolved

---

<div class="post-metadata">

**Author:** ![ego](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ego/32/31011_2.png) [@ego](https://discuss.circleci.com/u/ego)\
**Post date:** [September 23, 2016, 4:22pm UTC](https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965/2 "2016-09-23T16:22:19Z")

</div>

Ahhh!! I found out why this happens:

The cache is not saved until _after_ all the dependencies steps are completed. In fact it is the next stage after dependencies.

In my case I was trying to start a docker container in the dependencies \> post stage, which was failing, so I had not yet successfully managed to save my pip cache and every build it was having to install them from scratch.

Frustrating, but hopefully I can iterate faster now that it got through that stage successfully once.

I wish this was documented. I nearly gave up entirely.

---

<div class="post-metadata">

**Author:** ![FelicianoTech](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/felicianotech/32/21348_2.png) [@FelicianoTech](https://discuss.circleci.com/u/FelicianoTech)\
**Post date:** [September 23, 2016, 4:36pm UTC](https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965/3 "2016-09-23T16:36:43Z")

</div>

I’m glad you were able to figuring this out. Looks like we have this documented right in the configuration doc: [https://circleci.com/docs/configuration/#cache-directories](https://circleci.com/docs/configuration/#cache-directories)

Is there another place you were looking that’s missing this information? We’re constantly trying to improve our documentation.

---

<div class="post-metadata">

**Author:** ![ego](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ego/32/31011_2.png) [@ego](https://discuss.circleci.com/u/ego)\
**Post date:** [September 23, 2016, 4:41pm UTC](https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965/4 "2016-09-23T16:41:14Z")

</div>

I was reading here: [https://circleci.com/docs/how-cache-works/](https://circleci.com/docs/how-cache-works/) trying to understand why it wasn’t caching

I would have found it useful to have clearly stated that nothing is cached until _all_ dependency steps have completed successfully

---

<div class="post-metadata">

**Author:** ![FelicianoTech](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/felicianotech/32/21348_2.png) [@FelicianoTech](https://discuss.circleci.com/u/FelicianoTech)\
**Post date:** [September 23, 2016, 4:56pm UTC](https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965/5 "2016-09-23T16:56:30Z")

</div>

Thank you for the feedback. I’ve opened a GitHub Issue on the doc [here](https://github.com/circleci/circleci-docs/issues/338). We’ll work on making this clearer.

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/circleci/original/3X/5/1/511608acda8e85906e15909b57bd0a46c1dbe291.png) [@system](https://discuss.circleci.com/u/system)\
**Post date:** [June 18, 2018, 11:34am UTC](https://discuss.circleci.com/t/python-pip-dependencies-are-not-cached-reinstalled-every-time/6965/6 "2018-06-18T11:34:22Z")

</div>