Principles

Unit testing is a development process activity in which requests are made of software and the results are compared to expected values. This is valuable because tests can be run as sets against software to confirm that it is performing according to specification and that no failures emerge when maintainance is performed on established software. Unit testing is most valuable when it is automated by a unit test framework.

XUnit Testing

XUnit is NetKernel's unit test framework. XUnit will discover test included in modules and allow a user to run them from the XUnit application interface.

The test interface of a module must be configured to invoke the xunit accessor with a supplied testlist exposed as a resource on the public interface of a module (see below).

Unit tests are specified in a configuration file that adheres to this RelaxNG schema. When run, XUnit reads the configuration file and then executes all tests, reporting results back in either XML or HTML format.

In NetKernel, a unit test is simply a URI request dispatched to the address space of the host module. The XUnit framework organizes and dispatches each URI request and examines the result according to the test specification. There are a variety of ways to validate the response from a test. For example, an XPath query can test for certain returned values, the response can be compared against a schema, the metadata of the response can be tested, or a separate URI can be activated to perform any arbitrary amount of processing against the response.

Providing and Identifying a Module Test Interface

A module may present a test interface which can be automatically discovered by the test framework. The discovered tests are integrated and can be run from the developer tools. Here is a psuedo module definition showing how a test interface should be exported with a rewrite rule which maps the interface to the xunit accessor with the module's testlist passed as an argument.

Note: It is essential to write the mapping so that all arguments are captured and passed through to the xunit accessor or the tests cannot be interactively explored with the test framework.

Identifying the Test Interface

To be dynamically discovered the test interface must have an entrypoint. The test's entrypoint should have a category of xunit. Please see the entrypoint guide for details of how to export entrypoints for your module.

Unit Test Specification

Unit tests are specified in a testlist XML fragment. A unit test configuration has the following basic format:

In the testlist element, the desc element contains an HTML description followed by one or more test elements (that describe an individual test) or group elements (that reference a separately defined group of tests). The module and moduleVersion elements specify a specific module to be tested.

testlist Attributes

• @title - (optional) the name of the test (defaults to empty string)
• @output - (optional) indicate whether the results should be "xml" or "html" formatted. (default is "xml"). Tests with XML output can be executed in the Dynamic Testing Framework.
• @iterate - (optional) sets the number of times the tests will be iterated. Default is once.

testlist elements

• <desc> - (optional) A block of XHTML describing the tests.
• <test> - A single test (see below)
• <group> - A test group to recursively invoked [may have optional @title attribute] (see below)
• <module> - (optional) uri of module against which public URI address space the test URI will be invoked [may be overridden by invidual tests]. If not specified the test URI will be issued into the current URI address space (generally the typical mode of operation).
• <moduleVersion> - (optional) version number of module [may be overridden by individual tests]

The test element defines a single test:

Test elements

• <setup> - an optional URI that can be called to perform setup prior to running the test.
• <uri> - the URI to be invoked as the test.
• <teardown> - an optional URI that can be called to perform teardown after running the test.
• <assert> - optional assertion tests on the result (see below) - may have an optional @name attribute to distinguish the assertion block.
• <module> - optional uri of module against which public URI address space the test URI will be invoked
• <moduleVersion> - optional version number of module

Group element

• <group> - A test group to recursively invoked [may have optional @title attribute]
  • <uri> - the URI of the testlist to be recursed
• <module> - optional uri of module against which public URI address space the test URI will be invoked [may be overridden by invidual tests]. If not specified the test URI will be issued into the current URI address space (generally the typical mode of operation).
• <moduleVersion> - optional version number of module [may be overridden by individual tests]

Assertion Tests

When a test URI request has completed the XUnit engine will test the result against any/all specified assertions. Assertions may be of the following types

• <xpath> - An XPath expression
• <xsd> - The URI of an XML Schema against which to validate the response
• <rng> - The URI of an RelaxNG Schema against which to validate the response
• <schematron> - The URI of an Schematron Schema against which to validate the response
• <uri> - The URI of an assertion service which the XUnit engine will invoke with the test result as the param argument of the request. The assertion service must return a boolean result (either canonical boolean document or an IAspectBoolean).
• <minTime> - minimum time the test should take (milliseconds)
• <maxTime> - maximum time the test should take (milliseconds)
• <mimetype> - mimetype of result
• <expired/> - result should be expired
• <intermediate/> - result should be intermediate
• <contextSensitive/> - result should be context sensitive, ie the result may not be same if called from a different context (super stack)
• <exception/> - result should be an exception with the given id

Failures in assertions may be of two types. assertionException - indicates that the assertion test failed with an exception. assertionFailure - indicates that the assertion test returned false - assertionFailures will be accompanied by the name of the assertion block (if present) and the number of the assertion test in the assertion block which has failed.

URI Requests

XUnit is really just a sophisticated mapper which invokes sub-requests and tests the results. By default, the test URI will be issued as a request to the internal URI address space from wherever the xunit accessor is invoked. It is recommended that all test URIs should be absolute URIs - no guarantees are made which current working URI will be used to resolve relative URIs.

Using a Module URI

If an optional <module> URI is specified for a test, the test URI will be requested against the public URI interface of that module. If the testlist document specifies a module URI then all tests will be requested against the public URI interface of that module. An individual test may specify it's own module URI which will override the testlist module URI.

Example URIs

The URI of a test may be any NetKernel resolvable URI, here are some examples

• To test a resource exists: ffcpl:/path/to/my/resource.xml
• To start a DPML test harness use: active:dpml+operand@ffcpl:/path/to/my/process.idoc

Test Order and Execution State

The test engine will execute all tests specified in a testlist, including recursively included tests from groups. The tests are not guaranteed to be executed in order - no assumptions should be made about the order of execution of a series of tests. If necessary each test should be written to create any necessary state for the test - you must not assume address space state will be preserved from another test higher up the testlist.

XUnit Self Test

Xunit is self-testing - it's tests are automatically discovered and executable from the developer tools xunit tool.