Python versions with different unicode versions - need UCS4

python

#1

Hey,
I can see the following in our build environment:

$ /usr/bin/python -c 'import sys; print sys.maxunicode'
1114111

$ which python
/home/ubuntu/virtualenvs/venv-2.7.9/bin/python

$ python -c 'import sys; print sys.maxunicode'
65535

This indicated that system python was compiled with UCS4 unicode, while pyenv python were compiled with the default UCS2. UCS2 & UCS4 are not ABI compatible and stuff breaks when trying to run python code with C libs that was compiled under the other version.

Most Linux distros compile python with UCS4 (Ubuntu/Fedora/etc).

Any help would be appreciated.


#2

We would also like the pyenv python to be compiled with UCS4 - the difference is especially annoying when dealing with docker images - it prevents e.g. installing a binary wheel built in pyenv into a docker image.


#3

I filed a support ticket asking for UCS4 support - I didn’t see this issue before doing so and ran into the exact same issues trying to compile venv’s in Circle for AWS Lambda. Would VERY much like UCS4 unicode support in CircleCI’s python environments or at least added in any upcoming 2.7.11 environment.


#4

Thank you for letting us know about this issue.

Python is now compiled with UCS4 for all versions on our Ubuntu 14.04 ‘Trusty’ image.

You can switch your project to 14.04 from ‘Project settings > Build environment’.

ubuntu@box177:~$ python
Python 2.7.10 (default, Mar 15 2016, 20:34:18)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.maxunicode
1114111
ubuntu@box177:~$ pyenv local 3.5.1
ubuntu@box177:~$ python
Python 3.5.1 (default, Jan 29 2016, 18:22:19)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print (sys.maxunicode)
1114111 

#5