C++ Boost

Boost.Python

Building and Testing

Contents

Requirements
Building Boost.Python
Configuration
Configuration for Cygwin GCC from a Windows prompt
Results
Notes for Cygwin GCC Users
Notes for MinGW (and Cygwin with -mno-cygwin) GCC Users
Testing
Building your Extension Module
Build Variants
Building Using the Microsoft Visual Studio IDE

Requirements

Boost.Python version 2 requires Python 2.2 or newer. An unsupported archive of Boost.Python version 1, which works with versions of Python since 1.5.2, is available here.

Building Boost.Python

Normally, Boost.Python extension modules must be linked with the boost_python shared library. In special circumstances you may want to link to a static version of the boost_python library, but if multiple Boost.Python extension modules are used together, it will prevent sharing of types across extension modules, and consume extra code space. To build boost_python, use Boost.Build in the usual way from the libs/python/build subdirectory of your boost installation (if you have already built boost from the top level this may have no effect, since the work is already done).

Basic Configuration

You may need to configure the following variables to point Boost.Build at your Python installation. Variables can be either set in the environment or passed on the bjam command-line as -sname=value. Variable names are case-sensitive.
Variable Name Semantics Default Notes
PYTHON_VERSION The The 2-part python Major.Minor version number 2.4 Be sure not to include a third number, e.g. not "2.2.1", even if that's the version you have.
PYTHON_ROOT The root directory of your Python installation Windows: c:/Python(10*Version), e.g. c:/Python24
*nix/Cygwin: /usr		 On *nix, this should be the --prefix= directory used to configure Python when it was built and installed.
PYTHON_INCLUDES path to Python #include directories Autoconfigured from PYTHON_ROOT. Try the default before attempting to set it yourself.
PYTHON_LIB_PATH path to Python library object. Autoconfigured from PYTHON_ROOT. Try the default before attempting to set it yourself.

Configuration for Cygwin GCC from a Windows prompt

The following settings may be useful when building with Cygwin GCC (not MinGW) from a Windows command shell using a Windows build of bjam. If "bjam -v" does not report "OS=NT", these settings do not apply to you; you should use the normal configuration variables instead. They are only useful when building and testing with multiple toolsets on Windows using a single build command, since Cygwin GCC requires a different build of Python.
Variable Name Semantics Default
CYGWIN_PYTHON_[DEBUG_]VERSION The version of python being used under Cygwin. $(PYTHON_VERSION)
CYGWIN_PYTHON_[DEBUG_]ROOT *nix-style path containing the include/ directory containing python$(CYGWIN_PYTHON_[DEBUG_]VERSION)/python.h. $(PYTHON_ROOT)
CYGWIN_PYTHON_[DEBUG_]LIB_PATH path containing the user's Cygwin Python import lib libpython$(CYGWIN_PYTHON_[DEBUG_]VERSION).dll.a Autoconfigured from CYGWIN_PYTHON_ROOT
CYGWIN_PYTHON_[DEBUG_]DLL_PATH path containing the user's Cygwin Python dll (libpython$(CYGWIN_PYTHON_[DEBUG_]VERSION).dll) /bin

Notes for Cygwin GCC Users

If you are using Cygwin GCC to build extension modules, you must use a Cygwin build of Python. The regular Win32 Python installation that you can download from python.org will not work with your compiler because the dynamic linking conventions are different (you can use MinGW GCC if you want to build extension modules which are compatible with a stock Win32 Python). The Cygwin installer may be able to install an appropriate version of Python, or you can follow the traditional Unix installation process to build Python from source.

The special build configuration variables listed above make it possible to use a regular Win32 build of bjam to build and test Boost.Python and Boost.Python extensions using Cygwin GCC and targeting a Cygwin build of Python.

Notes for MinGW (and Cygwin with -mno-cygwin) GCC Users

If you are using a version of Python prior to 2.4.1 with a MinGW prior to 3.0.0 (with binutils-2.13.90-20030111-1), you will need to create a MinGW-compatible version of the Python library; the one shipped with Python will only work with a Microsoft-compatible linker. Follow the instructions in the "Non-Microsoft" section of the "Building Extensions: Tips And Tricks" chapter in Installing Python Modules to create libpythonXX.a, where XX corresponds to the major and minor version numbers of your Python installation.

Results

The build process will create a libs/python/build/bin-stage subdirectory of the boost root (or of $(ALL_LOCATE_TARGET), if you have set that variable), containing the built libraries. The libraries are actually built to unique directories for each toolset and variant elsewhere in the filesystem, and copied to the bin-stage directory as a convenience, so if you build with multiple toolsets at once, the product of later toolsets will overwrite that of earlier toolsets in bin-stage.

Testing

To build and test Boost.Python, start from the libs/python/test directory and invoke

bjam -sTOOLS=toolset test
This will update all of the Boost.Python v1 test and example targets. The tests are relatively verbose by default. To get less-verbose output, you might try 
bjam -sTOOLS=toolset -sPYTHON_TEST_ARGS= test
By default, PYTHON_TEST_ARGS is set to -v.

Building your Extension Module

Though there are other approaches, the smoothest and most reliable way to build an extension module using Boost.Python is with Boost.Build. If you have to use another build system, you should use Boost.Build at least once with the "-n" option so you can see the command-lines it uses, and replicate them. You are likely to run into compilation or linking problems otherwise.

The files required to build a Boost.Python extension module using bjam are the "local" files Jamfile, Jamrules, and boost_build.jam, and the boost/ and tools/build/v1/ subdirectories of your Boost tree. The latter directory contains the source code of the Boost.Build system, which is used to generate the correct build commands for your extension module. The 'v1' refers to Boost.Build version 1. Version 2 is pre-release and currently not ready for general use.

The libs/python/example/ project we're going to build is set up to automatically rebuild the Boost.Python library in place whenever it's out-of-date rather than just reusing an existing library, so you'll also need the Boost.Python library sources in boost/python/src/.

Note: Third-party package and distribution maintainers for various operating systems sometimes split up Boost's structure or omit parts of it, so if you didn't download an official Boost release you might want to browse our CVS structure to make sure you have everything you need, and in the right places.

The libs/python/example subdirectory of your boost installation contains a small example which builds and tests two extensions. To build your own extensions copy the example subproject and make the following two edits:

  1. boost-build.jam - edit the line which reads 
    boost-build ../../../tools/build/v1 ;
    so that the path refers to the tools/build/v1 subdirectory of your Boost installation.
  2. Jamrules - edit the line which reads 
    path-global BOOST_ROOT : ../../.. ;
    so that the path refers to the root directory of your Boost installation.

The instructions above for testing Boost.Python apply equally to your new extension modules in this subproject.

Build Variants

Three variant configurations of all python-related targets are supported, and can be selected by setting the BUILD variable:

The first two variants of the boost_python library are built by default, and are compatible with the default Python distribution. The debug-python variant corresponds to a specially-built debugging version of Python. On *nix platforms, this python is built by adding --with-pydebug when configuring the Python build. On Windows, the debugging version of Python is generated by the "Win32 Debug" target of the PCBuild.dsw Visual C++ 6.0 project in the PCBuild subdirectory of your Python distribution. Extension modules built with Python debugging enabled are not link-compatible with a non-debug build of Python. Since few people actually have a debug build of Python (it doesn't come with the standard distribution), the normal debug variant builds modules which are compatible with ordinary Python.

On many windows compilers, when extension modules are built with -D_DEBUG, Python defaults to force linking with a special debugging version of the Python DLL. Since this debug DLL isn't supplied with the default Python installation for Windows, Boost.Python uses boost/python/detail/wrap_python.hpp to temporarily undefine _DEBUG when Python.h is #included - unless BOOST_DEBUG_PYTHON is defined.

If you want the extra runtime checks available with the debugging version of the library, #define BOOST_DEBUG_PYTHON to re-enable python debuggin, and link with the debug-python variant of boost_python.

If you do not #define BOOST_DEBUG_PYTHON, be sure that any source files in your extension module #include <boost/python/detail/wrap_python.hpp> instead of the usual Python.h, or you will have link incompatibilities.

Building Using the Microsoft Visual Studio IDE

For the those of you who feel more comfortable in the IDE world, a workspace and project file have been included in the libs/python/build/VisualStudio subdirectory. It builds release and debug versions of the Boost.Python libraries and places them and the same directory as Jamfile build does, though the intermediate object files are placed in a different directory. The files have been created using Microsoft Visual C++ version 6, but they should work for later versions as well. You will need to tell the IDE where to find the Python Include/ and Libs/ directories. Under Tools>Options>Directories, add an entry for the Python include dir (i.e. c:/Python22/Include), and one for the Lib (i.e. c:/Python/Libs. Make sure it is Libs with an "s" and not just Lib).

Using the IDE for your own projects

Building your own projects using the IDE is slightly more complicated. Firstly, you need to make sure that the project you create as the right kind. It should be a "Win32 Dynamic-Link Library". The default one that Visual Studio 6 creates needs some modifications: turn on RTTI, and change the debug and release builds to use the respective debug and release Multithreaded DLL versions. You should probably turn off incremental linking too -- I believe it a bit flaky. If you do this, then change the "Debug Info" to "Program Database" to get rid of the Edit and Continue warning.

You'll need to add the Boost root directory under Tools>Options>Directories to get your code compiling. To make it link, add the above boost_python.dsp file to your workspace, and make your project depend upon it (under Project>Dependencies). You should be able to build now.

Lastly, go to the Project Settings>Debug Page and add the Python.exe as the executable for the project. Set a startup directory, and make sure that your current project's output dll, the boost_python.dll and the python22.dll are on the current PATH. If you have a python script that tests your dll, then add it in the "Program Arguments". Now, if all went well, you should be able to hit the Run (F5) button, and debug your code.

The Visual Studio project files are graciously contributed and maintained by Brett Calcott.

© Copyright David Abrahams 2002-2004. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided ``as is'' without express or implied warranty, and with no claim as to its suitability for any purpose.

Updated: 13 April 2004 (David Abrahams)