Running C++ Boost regressions with Boost.Build V2

This document describes how to run C++ Boost regression tests (using regression.py script), with Boost.Build V2.

Configuration

Unlike V1, configuration of toolset is not done by creating new toolset files. Rather, there's a single config file that should be edited. The file is called user-config.jam and must be placed to your home directory. For the exact meaning of "home directory", see http://boost.org/boost-build2/doc/html/bbv2/reference.html#bbv2.reference.init

The file should contain one entry for each toolset/version combination to be tested. Each entry should be in the form:

using <toolset-name> : <toolset-version> : <absolute name of compiler executable> ;

for example:

using msvc : 7.0 : "Z:/Programs/Microsoft Visual Studio/vc98/bin/cl.exe" ;

Note: use only forward slashes too, both on Unix and Windows. If the path contains spaces, the whole path should be quoted.

It's possible to specify additional options to compiler or linker:

using hp_cxx 
       : 65_042_tru64   # version
       :                # command not specified -- use default
       :                # options
     <compileflags>"-version V6.5-042"
     <linkflags>"-version V6.5-042 -use_non_shared_libcxx"
;

Another note is that while in V1 customized toolset is commonly named base_toolset-version, in V2 the version parameter to using should not include base toolset name. That is, using hp_cxx_64_042_tru64 above would be a mistake.

After declaring a toolset version, it's possible to use base_toolset-version to request this toolset when building.

You should also configure the location of Python, for example:

using python : 2.3 : C:/Python ;

Adjust the version and python root accordingly.

Running

First of all, obtain from CVS the current version of the regression.py script, located in the tools/regression/xsl_reports/runner directory. To use V2 when running regression.py, just add --v2 option. For example, to run tests and uploade results, run:

./regression.py --v2 --toolsets=hp_cxx-65_042_tru42 --runner=hp-tru42-V2

The names of toolsets passed to regression.py are constructed from name base toolset name and version passed to the using rule, separated with dash.

Please also add "-V2" suffix the the runner-id that you normally use, so that it's easy to see that you're using V2.

Note: On windows, use python regression.py .... Even if Python files are associated with Python interpreter, please write python explicitly in the command line -- there were reports of problems arising due to auto-running.

If you want to run the regression tests performing each step individually, you can do the following:

./regression.py cleanup
./regression.py get-source
./regression.py setup --v2 --toolsets=hp_cxx-65_042_tru42
./regression.py test --v2 --toolsets=hp_cxx-65_042_tru42 --bjam-options="warnings=on"
./regression.py collect-logs --comment comment.html --runner=hp-tru42-V2
./regression.py upload-logs --runner=hp-tru42-V2

where the --bjam-options allows you to pass in additional options to bjam/BBv2 and --comment allows you to add a custom HTML page about the runner.

Note: when running each step separately, step name should be the first thing on command line, before any options.

You can separate the toolsets used in the call for test, for example:

./regression.py test --v2 --toolsets=msvc-7.1
./regression.py test --v2 --toolsets=msvc-8.0

To perform a monitored build, you shouldn't use the --monitored option, but instead use --bjam-options="-l<s>" where s is the number of seconds a task should run before being terminated. For example, --bjam-options="-l600" will terminate any task running over 10 minutes.