Note

These instrctions are a modified version of the CPython devguide

4. Running & Writing Tests

Note

This document assumes you are working from an in-development checkout of Jython. If you are not then some things presented here may not work as they may depend on new features not available in earlier versions of Jython.

4.1. Running

The shortest, simplest way of running the test suite is the following command from the root directory of your checkout:

ant regrtest

To run a single test case, use the unittest module, providing the import path to the test case:

./python -m unittest -v test.test_abc.TestABC

If you have a multi-core or multi-CPU machine, you can enable parallel testing using several Python processes so as to speed up things:

   ./python -m test -j0

If you are running a version of Python prior to 3.3 you must specify the number
of processes to run simultaneously (e.g. ``-j2``).

.. _strenuous_testing:

Finally, if you want to run tests under a more strenuous set of settings, you
can run ``test`` as::

    ./python -bb -E -Wd -m test -r -w -uall

The various extra flags passed to Python cause it to be much stricter about
various things (the ``-Wd`` flag should be ``-W error`` at some point, but the
test suite has not reached a point where all warnings have been dealt with and
so we cannot guarantee that a bug-free Python will properly complete a test run
with ``-W error``). The ``-r`` flag to the test runner causes it to run tests in
a more random order which helps to check that the various tests do not interfere
with each other.  The ``-w`` flag causes failing tests to be run again to see
if the failures are transient or consistent.
The ``-uall`` flag allows the use of all available
resources so as to not skip tests requiring, e.g., Internet access.

You can also execute the ``Tools/scripts/run_tests.py`` script as  found in a
CPython checkout. The script tries to balance speed with thoroughness. But if
you want the most thorough tests you should use the strenuous approach shown
above.

4.2. Writing

Writing tests for Python is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. We try to have tests both for normal behaviour and for error conditions. Tests live in the Lib/test directory, where every file that includes tests has a test_ prefix.

One difference with ordinary testing is that you are encouraged to rely on the test.support module. It contains various helpers that are tailored to Python’s test suite and help smooth out common problems such as platform differences, resource consumption and cleanup, or warnings management. That module is not suitable for use outside of the standard library.

When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable.