| |||
|
MetaBoss Beginner's GuideDescription of the testing processOverviewIn systems built in accordance with Service Oriented Architecture concepts (and our Course Registration System is the fine example of such system) the testing process essentially comes down to making series of service calls against the system, which is in the known state, and comparing the actual result with the expected one. From this definition we can easily see what do we need to do:
In the MetaBoss testing methodology, the test procedure is run via Apache Ant and therefore the master file controlling an execution of the test procedure is the Ant file. Ant is very flexible and feature rich batch processor. Some tasks we use during testing are offered by Ant out-of-the-box. For example the SQL task allows to execute database instructions from within Ant script. We use it to automatically initialise the databases prior to test runs. In our example the test master file is located at ${metaboss_home}/examples/AlmaMater/Test/test.xml. The framework is described in more detail in the MetaBoss Testing Framework Guide. This chapter will explain in detail what steps are involved in testing our example using MetaBoss Testing Framework. IndexThe 'Find Free Teachers' test case descriptionThe 'Find Student Acquaintances' test case description Bringing the system into the known state Executing the test cases Running the test and interpreting the results A few words on automatic test validation The 'Find Free Teachers' test case descriptionWe will now have a look at the findFreeTeachers operation from the MiscellaneousQueries service. This operation is a query, it has no inputs and returns list of Teachers who is not Lecturer and not a Supervisor for any Course. Absence of the inputs makes this test a very simple one since there are less permutations we have to test for (ie. we do not need to call the operation many times with different inputs). In order to test the main functionality of this query we need to test if it returns all free Teachers and does not return any of the allocated Teachers. To do that we need to call it against the system populated with the dataset which has Teachers with following characteristics:
The above list of the required features in the dataset is by no means complete. In real life we may also wish to test some more boundary conditions, such as "No free Teachers" or "No allocated Teachers". The dataset will have to be more complex to accomodate all possible scenarios. However, we believe that what we have described is is enough for the purpose of this example. The 'Find Student Acquaintances' test case descriptionWe will now have a look at the findStudentAcquaintances operation from the MiscellaneousQueries service. This operation is a query, it requires Student Identification as an input and returns list of Teachers and Students who might be familiar with the given Student. Familiar Teachers are the ones who are Supervisors or Lecturers on one or more Courses the Student is enrolled into. Familiar Students are the ones who are enrolled in one or more same Courses the Student is enrolled into. This test case is not as simple as the previous one we have looked at. This is in part due to the complexity in the input Student Identification field. Students are identified by either InstanceId field or StudentNo field of the input StudentKey structure. (Every entity in the system has autogenerated InstanceId primary key field. In addition some entities have natural primary key fields. The entity Key structure contains all fields by which entity can be uniquely identified). All of this means that we will have to test invocation with the valid and invalid InstanceId and StudentNo input fields. In order to test the main functionality of this query we need to test if it returns all associated Teachers and Students and does not return any other ones. To do that we need to call it against the system populated with the dataset with Students having following characteristics:
The above list of the required features in the dataset is by no means complete. The dataset will have to be more complex to accomodate all possible scenarios. However, we believe that what we have described is is enough for the purpose of this example. Bringing the system into the known stateThe middleware we have created is stateless, which means that it does not keep in-memory state between service invocations (in other words it does not maintain client conversation context). This means that in order to bring our system to known state we only need to initialise underlying database with known dataset. The database is reinitialised in three steps:
Executing the test casesAfter the system is brouhgt to the known state, it is time to execute the test cases. Similarly to the dataset population step, this one uses the MetaBoss Scenario Runner utility which allows to execute series of service operations defined in XML scripts. The Ant task which performs the test looks like this: <MetaBossScenarioRunner scenarioname="AlmaMaterCRSSystemTest"
runname="testQueries"
scenariopath="${testscenario_dir}"
logdir="${testlogs_dir}"
classpathref="classpath.inprocess"/>
In this instance we run all the scripts from the ${metaboss_home}/examples/AlmaMater/Test/testscenario directory. Again, the ScenarioRunner will execute all test case script files found in this directory in the ascending alphabetical order. We have have found that the easiest way to control the order of execution is to use numeric prefix in the test case file names. We will now take a moment and look closer inside these scripts. As it was mentioned earlier, the scripts are XML files created in accordance with the MetaBoss ScenarioRunner XML schema in conjunction with automatically generated servicemodule schemas. You can learn more about ScenarioRunner XML schema here. The servicemodule schemas are generated as part of the domelements adapter source. Since we have two servicemodules in our system, we have two schemas. The CoursesDomainSupport servicemodule schema is located in com/almamater/crs/adapters/coursesdomainsupport/generic/domelements/CoursesDomainSupport.xsd and the Reporting servicemodule schema is located in com/almamater/crs/adapters/reporting/generic/domelements/Reporting.xsd. The fact that ScenarioRunner validates the Test cases against the schemas generated from the model means that all non backwards compatible changes to the model (and therefore to the schemas) will make affected test cases invalid and they wiil have to be fixed. We have found that this produces good 'early warning' signal and forces designers and developers to ensure backwards compatibility or pay the price for breaking it. All of the scripts in this directory are prepared by hand and look similar to the one below: <TestCasePlan xmlns="http://www.metaboss.com/XMLSchemas/MetaBoss/SdlcTools/SystemTester/1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
description="Test find free teachers service call from the Crs / Reporting functionality">
<TestCasePlan name="FindStudentAcquaintancesWithStudentNumber"
description="Executes the service with student number as the unique student identifier">
<OperationPlan>
<InputDocument>
<MiscellaneousQueries.FindStudentAcquaintancesInput xmlns="AlmaMater/CRS/Reporting">
<Student>
<StudentNo xmlns="AlmaMater/CRS/CoursesDomainSupport">1123</StudentNo>
</Student>
</MiscellaneousQueries.FindStudentAcquaintancesInput>
</InputDocument>
</OperationPlan>
</TestCasePlan>
<TestCasePlan name="FindStudentAcquaintancesWithStudentInstanceId"
description="Executes the service with instance id as the unique student identifier">
<OperationPlan>
<InputDocument>
<MiscellaneousQueries.FindStudentAcquaintancesInput xmlns="AlmaMater/CRS/Reporting">
<Student>
<InstanceId xmlns="AlmaMater/CRS/CoursesDomainSupport">1-1-rq</InstanceId>
</Student>
</MiscellaneousQueries.FindStudentAcquaintancesInput>
</InputDocument>
</OperationPlan>
</TestCasePlan>
</TestCasePlan>
The above script executes findStudentAcquaintances operation twice. The first invocation uses StudentNo attribute to identify the Student we are interested in (StudentNo attribute is declared as natural primary key in the model therefore it can be used to identify instances). The second invocation uses entity Instance Id to identify the Student we are interested in. Note how the whole document is declared as being governed by ScenarioRunner schema and some elements inside the document are declared as instances of the elemnts defined in CoursesDomainSupport or Reporting schemas. Running the test and interpreting the resultsThe directory ${metaboss_home}/examples/AlmaMater/Test contains the runtest.bat Windows batch file which invokes Ant and runs the test procedure described in this chapter. The files test.properties and domainconfig.xml in the same directory contain database connection information. If you wish to run these tests yourself you will have to modify database server details (computer name and port number) to point to your own MySQL installation. If your database name is not 'almamater_crs_courses' it will also have to be modified (Note that the database itself must be created manually - Ant script does not contain the task to do that). Why do we have two files with practically the same information? The test.properties file defines And properties. This file is included in the Ant script and the properties are used when 'sql' task is invoked (The task used to cleanup and initialise the database). The domainconfig.xml file is used by Tyrex transaction management framework. Tyrex is used by MetaBoss systems when they are deployed 'inprocess' (ie. outside J2EE container). It provides standalone implementation of the JTA (Java Transaction Architecture) mechanism. In other words test.properties used to initialise Ant whereas domainconfig.xml is used to initialise middleware. Upon completion of the test run, two log files are created in the ${metaboss_home}/examples/AlmaMater/Test/testlogs directory. The reason for having two log files is because we have invoked ScenarioRunner twice: first time to populate the database and second time to execute test scenario. Have a look at these log files. They list each operation which was invoked together with inputs to it and outputs from it. They certainly contain enough information for us to study the results and see if operations have functioned correctly (we know the dataset and the inputs, so it should not be a problem to validate the outputs). A few words on automatic test validationAutomatic test validation is the facility which allows to automate validation of the test case and not to rely 100% on human eyes. The automatic test validation falls outside the scope of this simple example. However it seems appropriate to mention them here. The MetaBoss Testing Framework offers two facilities for automation of the test case validation:
The good news is that both of theese facilities can be used simultaneously. If you wish to learn more about these features, please refer to the more comprehensive HatMaker example. |
|
Copyright © 2000-2005 Softaris Pty.Ltd. All Rights Reserved.
MetaBoss is the registered trademark of Softaris Pty.Ltd. Java, Enterprise JavaBeans, JDBC, JNDI, JTA, JTS, JCA and other Java related APIs are trademarks or registered trademarks of Sun Microsystems, Inc. MDA, UML, MOF, CORBA and IIOP are trademarks or registered trademarks of the Object Management Group. Microsoft, .NET and C# are trademarks or registered trademarks of the Microsoft Corporation. All other product names mentioned herein are trademarks of their respective owners. |