Summary

Have you ever wanted a newer version of Python on your Ubuntu install?

For example, Ubuntu 12.04.4 provides Python 2.7.3 and Python 3.2 versions, but you'd like to use a feature only available in 3.3+. Or perhaps, you need to test something on 2.6.6 to match what is avilable on Centos 6.

In this article, I'm going to show you how to install a newer version of Python that installs in your home directory without breaking your system's default installed version of Python.

You will also be able to switch between the system and your local versions, and still use tools such as pip and virtualenv.

I'm using Ubuntu 12.04.4 LTS, but you should be able to use this information in later versions of Ubuntu and it's various respins and derivatives. These instructions may also work on Debian as well.

Introducing Pyenv

pyenv lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well.

pyenv follows a family of home directory installing options for Python. In the past I've used pythonbrew and it's fork pythonz. I discovered pyenv about 6 months ago, and use it now for all my Python development.

I encourage visiting https://github.com/yyuu/pyenv for complete documentation on pyenv before continuing.

Getting Started

Whenver I install pyenv on a new system, I use the automatic installer. This automates the process of cloning the repository from github and such. Before you continue, you should make sure you have git and curl installed.

dbehnke@precise:~$ sudo apt-get install curl git-core

Running the Automatic Installer

dbehnke@precise:~$ curl https://raw.github.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash

Open up .bashrc located in your home directory with a text editor and add the following.

export PYENV_ROOT="${HOME}/.pyenv"

if [ -d "${PYENV_ROOT}" ]; then
  export PATH="${PYENV_ROOT}/bin:${PATH}"
  eval "$(pyenv init -)"
fi

Before changes will take place, you will need to exit your terminal and restart or type the following command

dbehnke@precise:~$ source ~/.bashrc

Verifying pyenv is installed properly

dbehnke@precise:~$ pyenv --version

pyenv 0.4.0-20140211-11-g5bbfcf7

Listing Available Python Versions

This is an example of what was available on March 2, 2014.

As you can see, there are many versions to choose from!

dbehnke@precise:~$ pyenv install --list

Available versions:
  2.4
  2.4.1
  2.4.2
  2.4.3
  2.4.4
  2.4.5
  2.4.6
  2.5
  2.5.1
  2.5.2
  2.5.3
  2.5.4
  2.5.5
  2.5.6
  2.6.6
  2.6.7
  2.6.8
  2.6.9
  2.6-dev
  2.7
  2.7.1
  2.7.2
  2.7.3
  2.7.4
  2.7.5
  2.7.6
  2.7-dev
  3.0.1
  3.1.3
  3.1.4
  3.1.5
  3.1-dev
  3.2
  3.2.1
  3.2.2
  3.2.3
  3.2.4
  3.2.5
  3.2-dev
  3.3.0
  3.3.1
  3.3.2
  3.3.3
  3.3.4
  3.3.5rc1
  3.3-dev
  3.4.0a2
  3.4.0a3
  3.4.0a4
  3.4.0b1
  3.4.0b2
  3.4.0b3
  3.4.0rc1
  3.4.0rc2
  3.4-dev
  anaconda-1.4.0
  anaconda-1.5.0
  anaconda-1.5.1
  anaconda-1.6.0
  anaconda-1.6.1
  anaconda-1.7.0
  anaconda-1.8.0
  jython-2.5.0
  jython-2.5.1
  jython-2.5.2
  jython-2.5.3
  jython-2.5.4-rc1
  jython-2.5-dev
  jython-2.7-beta1
  jython-dev
  miniconda-2.2.2
  miniconda-3.0.0
  miniconda3-2.2.2
  miniconda3-3.0.0
  pypy-1.5
  pypy-1.5-src
  pypy-1.6
  pypy-1.7
  pypy-1.7-dev
  pypy-1.8
  pypy-1.8-dev
  pypy-1.9
  pypy-1.9-dev
  pypy-2.0
  pypy-2.0.1
  pypy-2.0.1-src
  pypy-2.0.2
  pypy-2.0.2-src
  pypy-2.0-dev
  pypy-2.0-src
  pypy-2.1
  pypy-2.1-src
  pypy-2.2
  pypy-2.2.1
  pypy-2.2.1-src
  pypy-2.2-src
  pypy3-2.1-beta1
  pypy3-2.1-beta1-src
  pypy3-dev
  pypy-dev
  stackless-2.7.2
  stackless-2.7-dev
  stackless-3.2.2
  stackless-3.2-dev
  stackless-3.3-dev
  stackless-dev

Install Build Dependencies

First, since we will be compiling Python from source, your system needs to have the build dependencies. This is easy to do since 12.04.4 comes with Python 3.2 and the dependencies haven't changed much for 3.3 and later.

Here's how to install the build dependencies on an Ubuntu 12.04.4 system. We will pull in both 2.7 and 3.2 build dependencies.

dbehnke@precise:~$ sudo apt-get build-dep python2.7 python3.2

Reading package lists... Done
Building dependency tree
Reading state information... Done

The following NEW packages will be installed:
  autoconf blt blt-dev diffstat help2man libbluetooth-dev libdb5.1-dev libexpat1-dev libffi-dev libfontconfig1-dev libfreetype6-dev
  libgdbm-dev libjs-sphinxdoc libjs-underscore libncursesw5-dev libpthread-stubs0 libpthread-stubs0-dev libx11-dev libxau-dev
  libxcb1-dev libxdmcp-dev libxext-dev libxft-dev libxrender-dev libxss-dev libxss1 m4 python-jinja2 python-pygments python-sphinx
  quilt sharutils sphinx-common tcl8.5 tcl8.5-dev tk8.5 tk8.5-dev x11proto-core-dev x11proto-input-dev x11proto-kb-dev
  x11proto-render-dev x11proto-scrnsaver-dev x11proto-xext-dev xorg-sgml-doctools xtrans-dev xvfb
0 upgraded, 46 newly installed, 0 to remove and 0 not upgraded.
Need to get 15.9 MB of archives.

After this operation, 54.7 MB of additional disk space will be used.
Do you want to continue [Y/n]? Y

Additional Dependencies

Most of this is covered in the previous step, however sometimes it doesn't catch everything. Make sure to run this as well.

dbehnke@precise:~$ sudo apt-get install build-essential wget \
libreadline-dev libncurses5-dev libssl1.0.0 tk8.5-dev \
zlib1g-dev liblzma-dev

Installing Python

e.g. Python 3.3.4

dbehnke@precise:~$ pyenv install 3.3.4 -v > install-3.3.4.log
Downloading Python-3.3.4.tgz...
-> http://yyuu.github.io/pythons/9f7df0dde690132c63b1dd2b640ed3a6
Installing Python-3.3.4...
Installed Python-3.3.4 to /home/dbehnke/.pyenv/versions/3.3.4

Downloading setuptools-2.2.tar.gz...
-> https://pypi.python.org/packages/source/s/setuptools/setuptools-2.2.tar.gz
Installing setuptools-2.2...
Installed setuptools-2.2 to /home/dbehnke/.pyenv/versions/3.3.4

Downloading pip-1.5.4.tar.gz...
-> https://pypi.python.org/packages/source/p/pip/pip-1.5.4.tar.gz
Installing pip-1.5.4...
Installed pip-1.5.4 to /home/dbehnke/.pyenv/versions/3.3.4

If you want to see a verbose log of the install, open up install-3.3.4.log in a text editor.

Testing

$ which python

It should return ~/.pyenv/shims/python

Listing Python Versions

dbehnke@precise:~$ pyenv versions

* system (set by /home/dbehnke/.pyenv/version)
  2.7.6
  3.3.4

notice system is default.. system = os default installed python

Switch to Python Version for a shell session

e.g. using python 3.3.4

dbehnke@precise:~$ export PYENV_VERSION=3.3.4

dbehnke@precise:~$ pyenv versions
  system
  2.7.6
* 3.3.4 (set by PYENV_VERSION environment variable)

dbehnke@precise:~$ python
Python 3.3.4 (default, Mar  2 2014, 00:13:28)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Setting a Python Version for a Directory

e.g. for a test directory

dbehnke@precise:~$ mkdir test

dbehnke@precise:~$ cd test

dbehnke@precise:~/test$ pyenv local 3.3.4

dbehnke@precise:~/test$ pyenv versions
  system
  2.7.6
* 3.3.4 (set by /home/dbehnke/test/.python-version)

dbehnke@precise:~/test$ python

Python 3.3.4 (default, Mar  2 2014, 00:13:28)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

To show that we are still using the current version in the home directory.

test $ cd ..

dbehnke@precise:~$ python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Single use

You can also change versions for a single shell command.

dbehnke@precise:~$ PYENV_VERSION=3.4.0rc2 python

Python 3.4.0rc2 (default, Mar  2 2014, 16:04:06)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Rehash New Binaries

Anytime you install a package using "pip install" that also installs a helper executable, it usually won't work until you have pyenv rescan all the executables.

To accomplish this, run the rehash command.

dbehnke@precise:~$ pyenv rehash

Updating Pyenv Periodically

Since pyenv is installed by downloading from github, updating is very simple.

dbehnke@precise:~$ cd ~/.pyenv

dbehnke@precise:~$ git pull

dbehnke@precise:~/.pyenv$ git pull

Already up-to-date.

Advanced Use

Using pyenv to compile your a version of Python in your home directory may be enough. However, if you want to take it a step further and use virtualenv to sandbox your packages, I suggest using the pyenv-virtualenv plugin.

Virtualenv and Virtualenvwrapper will be covered in more depth in a future article.

However, if you are already familiar with virtualenv, please look at yyuu's pyenv-virtualenv plugin documentation at https://github.com/yyuu/pyenv-virtualenv

Here is a brief summary of how to use the pyenv-virtualenv plugin.

Installation

(This may already be installed if you followed the automated installer instructions for pyenv)

$ git clone git://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv

Creating a virtualenv is similar to installing a new version of Python.

e.g. Creating a virtualenv based on 3.3.4 called my334

dbehnke@precise:~$ pyenv virtualenv 3.3.4 my334

List the versions and you will see my334 is in the list

dbehnke@precise:~$ pyenv versions

* system (set by /home/dbehnke/.pyenv/version)
  3.3.4
  my334

Changing to a virtualenv is also similar to how we change a python version.

dbehnke@precise:~$ export PYENV_VERSION=my334

dbehnke@precise:~$ pyenv versions
  system
  3.3.4
* my334 (set by PYENV_VERSION environment variable)

Conclusions

Python can be installed almost anywhere with pyenv provided you have the proper prerequites installed. You can have multiple versions installed for testing with older systems. Release canidates are also available! You can use tools such as virtualenv to extend the sandboxing.


Comments

comments powered by Disqus