# Testing in different environments

**URL:** <https://discuss.circleci.com/t/testing-in-different-environments/450>\
**Category:** Programming Languages\
**Tags:** multiple-versions, python\
**Created:** [November 10, 2015, 9:17pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450 "2015-11-10T21:17:38Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![hanneshapke](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/hanneshapke/32/267_2.png) [@hanneshapke](https://discuss.circleci.com/u/hanneshapke)\
**Post date:** [November 10, 2015, 9:17pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/1 "2015-11-10T21:17:38Z")

</div>

I would like to test a Django package in different Django environments.

`Travis CI` has the convenient option of defining the Python and Django version which generates the permutations of the tests.

I was looking for a similar setup option for the yaml files in CircleCI, but I can’t find this option in the documentation.

**How can I test a package on CircleCI within different environments in the same test run?**

---

<div class="post-metadata">

**Author:** ![DevendraMeena1](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/devendrameena1/32/302_2.png) [@DevendraMeena1](https://discuss.circleci.com/u/DevendraMeena1)\
**Post date:** [November 11, 2015, 6:46am UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/2 "2015-11-11T06:46:30Z")

</div>

Tox is a tool for automate testing in Python. It does so by reading  
a tox.ini file where we specify the environments we want to test, and it will create a brand new  
virtual environment for that setup and run the test suite against it.  
Let’s start by installing tox and add it to our requirements.txt file.

(django-foobar):~/django-foobar$ pip install tox  
(django-foobar):~/django-foobar$ echo “tox” \>\> requirements.txt

Create a new file tox.ini in the projects working directory and paste the content below.

[tox]  
envlist =  
py27-django1.6,  
py34-django1.8

[django1.6]  
deps =  
Django\>=1.6,\<1.7

[django1.8]  
deps =  
Django\>=1.8,\<1.9

[testenv]  
commands =  
python {toxinidir}/setup.py test

[testenv:py27-django1.6]  
basepython = python2.7  
deps =  
{[django1.6]deps}

[testenv:py34-django1.8]  
basepython = python3.4  
deps =  
{[django1.8]deps}

As you can see we have a [tox] block in which we defines a list of environments we want to test.  
Next, we define some blocks for the different Django versions we want to test. The [testenv] block  
defines the command we want to run, and finally we combine the [testenv] with the different version  
blocks in order to fire up the test runner.

---

<div class="post-metadata">

**Author:** ![alexey](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/alexey/32/42_2.png) [@alexey](https://discuss.circleci.com/u/alexey)\
**Post date:** [November 11, 2015, 1:56pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/3 "2015-11-11T13:56:55Z")

</div>

One option would be to to use tox if you would like to run the test sequentially on two (or more) Python versions. Some customers have reported success with the following configuration:

tox.ini

```ini
[tox]
envlist = py26,py34
[testenv]
commands=pip install -e .
         pip install -r requirements-dev.txt 
         python setup.py test 
install_command=pip install --process-dependency-links --allow-external --allow-unverified {opts} {packages}

```

circle.yml

```yaml
dependencies:
   pre:
     - pip install tox

test:
   override:
     - tox

```

Another option would be to simply change the Python version, either during the build or on one of the containers in a parallel build. We use `pyenv` to manage Python versions, so you could do something like this if you are using a single container:

```yaml
machine:
  python:
    version: 2.6.8
​
test:
  override:
    - make test
    - pyenv global 2.7.9
    - make test

```

Alternatively, you can test both versions in parallel—e.g. set up a parallel build and make a few build containers run different versions of Python.

We add an env var called $CIRCLE\_NODE\_INDEX when a build runs with more than one container. This means that you can just make the Python choice with a simple if condition in the dependencies: pre step. Mind that the machine section cannot handle multiple configurations, so you could do something like this:

```yaml
machine:
  python:
    version: 2.6.8
​
dependencies:
  pre:
    - if [$CIRCLE_NODE_INDEX == "1"] ; then pyenv global 2.7.9 ; fi
​
test:
  override:
    - make test: # note the colon
        parallel: true

```

​  
This will configure Python 2.7.9 instead of 2.6.8 only on the second container (index starts at 0).

---

<div class="post-metadata">

**Author:** ![hanneshapke](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/hanneshapke/32/267_2.png) [@hanneshapke](https://discuss.circleci.com/u/hanneshapke)\
**Post date:** [November 11, 2015, 8:29pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/4 "2015-11-11T20:29:22Z")

</div>

Thank you for your replies! Great community!

---

<div class="post-metadata">

**Author:** ![ckcollab](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ckcollab/32/957_2.png) [@ckcollab](https://discuss.circleci.com/u/ckcollab)\
**Post date:** [January 12, 2016, 6:51am UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/5 "2016-01-12T06:51:49Z")

</div>

> [@alexey](#):
>
> test:  
> override:  
> - make test  
> - pyenv global 2.7.9  
> - make test

Using this method for `3.5.0` + `py.test` and it didn’t seem to change which version of Python was used?

Thanks for any help, appreciate it!

---

<div class="post-metadata">

**Author:** ![alexey](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/alexey/32/42_2.png) [@alexey](https://discuss.circleci.com/u/alexey)\
**Post date:** [January 13, 2016, 9:27am UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/6 "2016-01-13T09:27:44Z")

</div>

Could you please post the exact steps you have in your `circle.yml`? I will try to reproduce it and figure out what could have gone wrong. Thanks.

---

<div class="post-metadata">

**Author:** ![ckcollab](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ckcollab/32/957_2.png) [@ckcollab](https://discuss.circleci.com/u/ckcollab)\
**Post date:** [January 14, 2016, 4:29am UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/7 "2016-01-14T04:29:45Z")

</div>

```auto
machine:
  python:
    version: 2.7.5

dependencies:
  override:
    - pip install -r requirements.txt
test:
  override:
    - pyenv global 3.5.0
    - py.test

```

With that, py.test ran with 2.7.5

Repo here: [https://github.com/ckcollab/cassiopeia](https://github.com/ckcollab/cassiopeia)

Tests here: [https://circleci.com/gh/ckcollab/cassiopeia](https://circleci.com/gh/ckcollab/cassiopeia)

---

<div class="post-metadata">

**Author:** ![alexey](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/alexey/32/42_2.png) [@alexey](https://discuss.circleci.com/u/alexey)\
**Post date:** [January 15, 2016, 5:45pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/8 "2016-01-15T17:45:32Z")

</div>

I was able to reproduce this—looks like the reason is the `virtualenv` that we create by default to ensure the consistency of the Python environment.

You could do something like this to work around it:

```auto
test:
  override:
    - deactivate; pyenv global 3.5.0; py.test

```

Could you please try that out?

---

<div class="post-metadata">

**Author:** ![ckcollab](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/ckcollab/32/957_2.png) [@ckcollab](https://discuss.circleci.com/u/ckcollab)\
**Post date:** [January 22, 2016, 2:37am UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/9 "2016-01-22T02:37:48Z")

</div>

I got…

```auto
deactivate; pyenv global 3.5.0; py.test

bash: line 1: py.test: command not found deactivate; pyenv global 3.5.0; py.test returned exit code 127

```

Nothing seems to be working with `pyenv global 3.5.0`, tried these as well (no `deactivate`):

```auto
pyenv global 3.5.0; py.test
pyenv global 3.5.0 && py.test
# these actually run, but as 2.7.5

```

ALSO, side problem, Python 2.7 on Circle seems to not work with urllib + https? See this thread: [Python 2 not "built with ssl module"?](https://discuss.circleci.com/t/python-2-not-built-with-ssl-module/1401)

---

<div class="post-metadata">

**Author:** ![vhermecz](https://avatars.discourse-cdn.com/v4/letter/v/c37758/32.png) [@vhermecz](https://discuss.circleci.com/u/vhermecz)\
**Post date:** [February 20, 2016, 7:59pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/10 "2016-02-20T19:59:47Z")

</div>

First of all, thanks for the detailed reply, that is a nice workaround.  
But I wouldnt call it a solution. It is a terrible hack.  
Are there any plans, to support this scenario directly from circle.yml?  
While for a specific end-product this is a rare scenario, a FOSS usually requires testing against different configurations.  
Thank you,  
vhermecz

---

<div class="post-metadata">

**Author:** ![RealSalmon](https://avatars.discourse-cdn.com/v4/letter/r/3d9bf3/32.png) [@RealSalmon](https://discuss.circleci.com/u/RealSalmon)\
**Post date:** [February 21, 2016, 1:18am UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/11 "2016-02-21T01:18:04Z")

</div>

I struggled with this a little bit today, but ended up with a solution that seems not so bad after all. I did a quick write up of it here . . .

[https://ben.fogbutter.com/2016/02/20/testing-multiple-python-versions-on-circleci.html](https://ben.fogbutter.com/2016/02/20/testing-multiple-python-versions-on-circleci.html)

I hope this works for others as well!

---

<div class="post-metadata">

**Author:** ![sschuberth](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sschuberth/32/15469_2.png) [@sschuberth](https://discuss.circleci.com/u/sschuberth)\
**Post date:** [February 17, 2017, 9:46pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/12 "2017-02-17T21:46:21Z")

</div>

Using pyenv to switch Python versions only worked for me after I did

```
machine:
  pre:
    - brew install pyenv
    - echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
    - source ~/.bash_profile
```

---

<div class="post-metadata">

**Author:** ![j-funk](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/j-funk/32/15790_2.png) [@j-funk](https://discuss.circleci.com/u/j-funk)\
**Post date:** [May 5, 2017, 8:02pm UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/13 "2017-05-05T20:02:40Z")

</div>

If you’re using CCI v2.0 this works great for me, Docker ftw…

```
version: 2

jobs:
  build:
    working_directory: ~/your_thing
    docker:
      - image: themattrix/tox
    steps:
      - checkout
      - run: tox

```

More information…

> **[themattrix/docker-tox](https://github.com/themattrix/docker-tox)**
>
> docker-tox - Docker base image for running tox with Python 2.6, 2.7, 3.3, 3.4, 3.5, 3.6, PyPy, and PyPy3.

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/circleci/original/3X/5/1/511608acda8e85906e15909b57bd0a46c1dbe291.png) [@system](https://discuss.circleci.com/u/system)\
**Post date:** [June 18, 2018, 11:33am UTC](https://discuss.circleci.com/t/testing-in-different-environments/450/14 "2018-06-18T11:33:01Z")

</div>