circleci build fails when tox specifies multiple python versions, *and sometimes uses the wrong python version*

[stavxyz]

The error we see from tox in the circle build logs says:

ERROR:   py32: InvocationError: Failed to get version_info for python3.2: pyenv: python3.2: command not found

The `python3.2' command exists in these Python versions:
  3.2
  3.2.5

EDIT: Got past this error, now the problem is tox is finding the wrong python binaries to use for a couple of the virtual environments. By this I mean e.g. specifying py31 with tox results in tox actually using python 2.7.6. Skip to the comment on the latest failure description.

This is what appears to be the real issue: pyenv/pyenv-virtualenv#21

I want to come up with a solution that "just works", preferably without hacking up the tox.ini file. I have a hunch that if we could adjust the circleci build environment configuration so that it doesn't create the virtualenv for us, this would work. We can leave the virtualenv creation to tox?

---

The problem definition I have so far:

CircleCI uses pyenv and the problem seems to be the virtualenv created by pyenv and tox not playing nice together. Specifying a python version in the circle.yml means circle will create and enable a python virtualenv (really a pyenv virtualenv?) of the specified version in your build environment. Apparently there is a difference between pyenv virtualenv and virtualenv? Some details on that here:

From pyenv/pyenv-virtualenv#21 (comment)

More documentation around this might be a good idea in the future. Specifically, something that says that when you activate a pyenv virtualenv that all the shims for other Python versions aren't available.

A comment on pyenv + tox:

From pyenv/pyenv@17c153b:

NOTE: You can activate multiple versions at the same time, including mulitple versions of Python2 or Python3 simultaneously. This allows for parallel usage of Python2 and Python3, and is required with tools like tox. For example, to set your path to first use your system Python and Python3 (set to 2.7.9 and 3.4.2 in this example), but also have Python 3.3.6, 3.2, and 2.5 available on your PATH, one would first pyenv install the missing versions, then set pyenv global system 3.3.6 3.2 2.5. At this point, one should be able to find the full executable path to each of these using pyenv which, e.g. pyenv which python2.5 (should display $PYENV_ROOT/versions/2.5/bin/python2.5), or pyenv which python3.4 (should display path to system Python3).

http://dtor.com/halfire/2015/01/23/pyenv_tox_can_get_along.html

---

Here are some folks who may have found a workaround:
coala/coala#653

---

Possibly related issues:

pyenv/pyenv#34
coala/coala#653
pyenv/pyenv#206
pyenv/pyenv#348

Other links:

• CircleCI's docs on Python: https://circleci.com/docs/language-python

[stavxyz]

almost fixed

Adding the following command to the circle.yaml fixes the issue, only partially.

pyenv local 3.4.3 3.3.3 3.2.5 3.1.5 2.7.10 2.6.8

/~https://github.com/yyuu/pyenv/blob/master/COMMANDS.md#pyenv-local-advanced

See builds on CircleCI: https://circleci.com/gh/samstav/circleci-python-sandbox

So, if you were only testing python 2.7, 3.3, and 3.4 in your project, you need to tell pyenv which explicit version of python you want them to map to, just for those 3:

pyenv local 2.7.10 3.3.3 3.4.3

Preferably the version of python you need to test is listed here:

https://circleci.com/docs/environment#python

Otherwise you'll have pyenv install them.

At the time of writing this, I ssh'd into the circleci build box and ran pyenv versions:

$ pyenv versions
* system (set by /home/ubuntu/.pyenv/version)
  2.6.6
  2.6.8
  2.7
  2.7.10
  2.7.3
  2.7.4
  2.7.5
  2.7.6
  2.7.7
  2.7.8
  2.7.9
  3.1.5
  3.2
  3.2.5
  3.3.0
  3.3.2
  3.3.3
  3.4.0
  3.4.1
  3.4.2
  3.4.3
  pypy-2.2.1
  pypy-2.3.1
  pypy-2.4.0
  pypy-2.5.0

My almost-working circle.yml has this:

dependencies:
  post:
    - pyenv local 3.4.3 3.3.3 3.2.5 3.1.5 2.7.10 2.6.8

[stavxyz]

Reopening... turns out this wasn't working quite perfectly. I added some validation to ensure that the python version being used in the tox run is exactly the version expected based on the versions supplied to pyenv local in the circle.yml.

Go find the latest failure: https://circleci.com/gh/samstav/circleci-python-sandbox/tree/master

I get failures for python2.7 (it uses 2.7.6 instead of 2.7.10) and python3.1 (it uses 2.7.6 instead of 3.1.5)

[stavxyz]

I think if I could 1) get tox installed outside of the virtualenv created by circle, and/or 2) skip the step in circle's machine configuration which creates that initial virtualenv automatically or 3) if I can't do (2), just run deactivate, then somehow do (1)

....

Then this would "just work". There is something funky going on in the virtual environment created by pyenv. The versions I set by running pyenv local are being ignored (or they are not high enough in precedence) sometimes. In the current failure it is for python2.7 (uses 2.7.6 instead of 2.7.10) and python3.1 (it uses 2.7.6 instead of 3.1.5) but everything works fine for the py26, py32, py33 and py34 runs.

[stavxyz]

Need to dig in here: https://bitbucket.org/hpk42/tox/src/3ed5dc353a99acf57859a2dd265b5c2e973480e3/tox/interpreters.py?at=default&fileviewer=file-view-default

[stavxyz]

Fixed this with a tox plugin:
/~https://github.com/samstav/tox-pyenv
https://pypi.python.org/pypi/tox-pyenv