Running Boost Regression Tests

Providing detailed information about your environment

Once you have your regression results displayed in the Boost-wide reports, you may consider providing a bit more information about yourself and your test environment. This additional information will be presented in the reports on a page associated with your runner ID.

By default, the page's content is just a single line coming from the comment.html file in your regression.py directory, specifying the tested platform. You can put online a more detailed description of your environment, such as your hardware configuration, compiler builds, and test schedule, by simply altering the file's content. Also, please consider providing your name and email address for cases where Boost developers have questions specific to your particular set of results.

Incremental runs

You can run regression.py in incremental mode [4] by simply passing it an identically named command-line flag:

python regression.py ... --incremental

Dealing with misbehaved tests/compilers

Depending on the environment/C++ runtime support library the test is compiled with, a test failure/termination may cause an appearance of a dialog window, requiring human intervention to proceed. Moreover, the test (or even of the compiler itself) can fall into infinite loop, or simply run for too long. To allow regression.py to take care of these obstacles, add the --monitored flag to the script invocation:

python regression.py ... --monitored

That's it. Knowing your intentions, the script will be able to automatically deal with the listed issues [5].

Getting sources from CVS

If you already have a CVS client installed and configured, you might prefer to get the sources directly from the Boost CVS repository. To communicate this to the script, you just need to pass it your SourceForge user ID using the --user option; for instance:

python regression.py ... --user=agurtovoy

You can also specify the user as anonymous, requesting anonymous CVS access. Note, though, that the files obtained this way tend to lag behind the actual CVS state by several hours, sometimes up to twelve. By contrast, the tarball the script downloads by default is at most one hour behind.

Integration with a custom driver script

Even if you've already been using a custom driver script, and for some reason you don't want regression.py to take over of the entire test cycle, getting your regression results into Boost-wide reports is still easy!

In fact, it's just a matter of modifying your script to perform two straightforward operations:

1. Timestamp file creation needs to be done before the CVS update/checkout. The file's location doesn't matter (nor does the content), as long as you know how to access it later. Making your script to do something as simple as echo >timestamp would work just fine.

2. Collecting and uploading logs can be done any time after process_jam_log' s run, and is as simple as an invocation of the local copy of $BOOST_ROOT/tools/regression/xsl_reports/runner/collect_and_upload_logs.py script that was just obtained from the CVS with the rest of the sources. You'd need to provide collect_and_upload_logs.py with the following three arguments:

   --locate-root   directory to to scan for "test_log.xml" files
   --runner        runner ID (e.g. "Metacomm")
   --timestamp     path to a file which modification time will be used 
                   as a timestamp of the run ("timestamp" by default)
   

   For example, assuming that the run's resulting binaries are in the $BOOST_ROOT/bin directory (the default Boost.Build setup), the collect_and_upload_logs.py invocation might look like this:

   python $BOOST_ROOT/tools/regression/xsl_reports/runner/collect_and_upload_logs.py 
      --locate-root=$BOOST_ROOT/bin
      --runner=Metacomm
      --timestamp=timestamp