My manager (www.nathanr.net) recently shared the following technique that we use to manage our python instances, I thought it was so good, I wanted to share it with you.

No longer stress about what version of python is installed with your Linux distribution.

Python source + virtualenv + pip is your savior.

Virtualenv (http://pypi.python.org/pypi/virtualenv) is a bit like a vm for a python install, it takes a vanilla python build (typically from source) and allows you to "activate" it (replacing your default python binaries and libraries with the ones within the virtualenv instance.

This has the benefit of freeing you from the outdated python binaries found in most Linux distributions, and allowing you to tailor specific library configurations for each environment.

Pip (http://pypi.python.org/pypi/pip/1.1) is a tool for installing and managing python packages (a bit like CPAN for python). We can use this within the virtualenv instance to manage our installed libraries.

By default it queries the online python package index (PyPI - http://pypi.python.org/pypi).

Building python and installing virtualenv into it

Before running the following, you will need to download the python source code (http://www.python.org/download/) and virtualenv (http://pypi.python.org/pypi/virtualenv).

#install dev libraries (SLES SP2) #SUSE build tool pattern (same as build-essential on Ubuntu) zypper install zypper install -t pattern Basis-Devel zypper install libbz2-devel zypper install readline-devel zypper install ncurses-devel zypper install libopenssl-devel zypper install libxslt-devel

#build python

tar -xjf Python-2.7.3.tar.bz2

cd Python-2.7.3/

./configure --prefix=/usr/local/python-2.7.3

make

make install

ln -s python-2.7.3 python-2.7

ln -s python-2.7 python

#install virtualenv tar -xzf ~/Downloads/virtualenv-1.7.1.2.tar.gz cd virtualenv-1.7.1.2/

/usr/local/python/bin/python setup.py install

Ok, we now have a Python 2.7 install with virtualenv in it. Now, create a virtualenv for our development and make it your working virtual environment:

#set up virtualenv in environment-name /usr/local/python/bin/virtualenv environment-name New python executable in environment-name/bin/python Installing setuptools............done. Installing pip...............done.

#activate the virtual environment cd environment-name/. bin/activate (environment-name)user@host:~/tmp/environment-name>

Installing python libraries into the virtualenv instance

Note that some libraries in the online pip repositories are somewhat dated, to ensure that we have the correct versions we maintain a local cache (which we keep checked into our mercurial repository).

pip install --requirement=~/work/libraries/python/python-pip-freeze.txt --find-links=file://$HOME/work/libraries/python/cache --no-index

Where the cache folder contains the tar.gz files for all of the libraries contained in the python-pip-freeze.txt file (this file can easily be regenerated using the pip freeze command).

In our case the file contains the following libs:

CherryPy==3.2.2 Mako==0.7.0 MarkupSafe==0.15 PyXB-base==1.1.3 SQLAlchemy==0.7.7 Twisted==12.0.0 psycopg2==2.4.5 pyOpenSSL==0.13 reportlab==2.5 suds==0.4 wsgiref==0.1.2 zope.interface==3.8.0 pyclamdplus==1.0b2 python-magic==0.4.2 six==1.1.0 python-dateutil==2.1 lxml==2.3.4 python-daemon==1.5.5

Pulling it all together

We use the following helper script to create a new virtualenv environment and deploy our required libraries into this environment:

**#!/bin/bash

Need two parameters:

1. Which installed python to use

2. Where the virtualenv should be

if [[ $#!= 2 ]]; then echo "Usage: build-python-env.sh " echo "eg. For the distribution shipped python, you might do: \"build-python-env.sh /usr pyenv-test\"" exit 1 fi ZKDIR=dirname $(readlink -f $0) echo $ZKDIR $1/bin/virtualenv $2. $2/bin/activate pip install --requirement=$ZKDIR/libraries/python/python-pip-freeze.txt --find-links=file://$ZKDIR/libraries/python/cache --no-index **