For experimenting with novel CPython features, you can quickly set up an isolated environment. This post shows you how to do that.

I did this on Ubuntu 20.04 LTS with standard build tools installed, but the same instructions probably work more generally.

Compile python

Download latest CPython sources

git clone git@github.com:python/cpython.git

Change to a stable branch instead of main (so that we don’t have to build all libraries from source); here we’re using 3.11, the latest stable branch:

git checkout 3.11

To run the build, use the following (standard) commands.

configure --enable-optimizations
make
make test

--enable-optimizations does some instrumentation first, runs a demo workload, and then compiles again using deemed best compiler options.

make test may not be necessary, but probably not a bad idea. For me, test_ssl fails, but I’ll ignore that for now.

Note: If you want several builds to compare, you need to have a full copy of the source (cpython root) folder; you can build in a subfolder, but that doesn’t change that all pythons share the Lib folder and hence only the latest compile works correctly. This seems to remain the case even with a venv that isolates the installed packages. You cannot run a Python version if you change the git checkout to a different version; the build still uses the Lib subfolder from the cpython repo.

pip bootstrap

The better option: Create a virtual environment, see below.

So far, the compilation generated a naked python executable that is just the Python interpreter. For almost anything interesting, we will have to install packages, and the most convenient way for that is pip.

Python already comes with a bootstrap module to do that (https://pip.pypa.io/en/stable/installation/):

./python -m ensurepip --upgrade

That’s it! Now you can run

./python -m pip install numpy pandas

etc. to install packages. These all get installed into the system wide folder as

./python -m pip show pandas

reveals.

Create a venv

A virtual environment is a folder with all Python needs, isolated from other installations.

./python -m venv my-python

generates a virtual environment in the subfolder

source my-python/bin/activate

make this the active venv for the current running shell. Check python --version to see if it worked.

From now on, you can use python instead of ./python and pip directly instead of ./python -m pip etc.

Moreover, a call to

python -m pip show pandas

reveals that these are now local to your project (the venv my-python really), and that is much better isolation.

Why a stable branch?

CPython is reasonably easy and quick to compile, so why not simply work with the current main branch? The main reason (no pun intended) is to easily be able to install any Python packages with pip without much hassle. For major releases (like 3.11), PyPi has precompiled “wheels” of many popular packages and so installing them does not need all their build dependencies installed and is very quick.

Since Python version jumps often affect the C API, many libraries also lag a bit behind CPython main and will not easily be usable with the development branch.