A Test runs successfully of:

Benchmark is a Test, its result is a time.

First, we specify parents Tests.

To do this, just overload Action method with needed inputs and outputs, after, with the Java reflection mechanism we found the first good Action method to execute the Test.

Mechanism in details

In standalone mode the test runs with this method:

 void action() throws Exception;  

In interlinked mode, the developer must to add a similar method in his code Test:

  A action(B toto, C tutu, …) throws Exception; 

Where toto is the result output of the first parent of this test and tutu the result output of second parent, … A is the type of result output of this Test.

Reflection code

Find the first action method:

 Method[] methods = getClass().getMethods();
            Method actionWithParams = null;
            for (int i = 0; i < methods.length; i++) {
                if ((methods[i].getName().compareTo('action') == 0) &&
                        (methods[i].getReturnType().getName().compareTo('void') != 0)) {
                    actionWithParams = methods[i];
                    break;
                }
            }

Array of params type:

            if (actionWithParams != null) {
                Object[] args = null;
                if (tests != null) {
                    args = new Object[tests.length];
                    for (int i = 0; i < tests.length; i++)
                        args[i] = tests[i].getOut();
                } else {
                    args = new Object[1];
                    args[0] = null;
                }

Call the method:

 out = actionWithParams.invoke(this, args); 

In Manager you can modify this by:

yourManager.setVerbatim(true/false)  

If Verbatim value is:

By default Verbatim is at false

This option has no effect on XML and HTML exports.

To see the value of Verbatim:

  yourManager.isVerbatim() 

See Section 48.5, “Configuration File” for more detail to configure result output through the file descriptor.

To add logs in your Test code it is very easy: you can directly use the variable logger or the getter getLogger(). This is a org.apache.log4j.Logger

In your Test code just add logs like this:

if (logger.isDebugEnabled())
     logger.debug('A debug message ...');

For more information about use logger see the log4J manual.

By default all logs with a level higher than INFO are written in $HOME/tests.log.

But you can configure the format and plac where you want to get logs.

The log4j environment is fully configurable programmatically. However, it is far more flexible to configure log4j using configuration files. Currently, configuration files can be written in XML or in Java properties (key=value) format.

You can also configure the logger by the Section 48.5, “Configuration File”.

Use default configuration of log4J. Add this code Manager constructor:

// Set up a simple configuration that logs on the console.
 BasicConfigurator.configure();

An another example to write logs in an HTML file:

  public YourManager() {
        super('Function calls', 'Alpha version');
        HTMLLayout layout = new HTMLLayout();
        WriterAppender appender = null;
        try {
            FileOutputStream output = new FileOutputStream(
                    '/net/home/adicosta/output2.html');
            appender = new WriterAppender(layout, output);
        } catch (Exception e) {
        }
        logger.addAppender(appender);
        logger.setLevel(Level.DEBUG);
    }

For more information about logger configuration see the log4J manual.

It is very simple to use it. Just do like this example:

You have a private variable in your Manager or Test:

 private int packetSize = 1024;

First add a setter of which it take a String in input:

     public void setPacketSize(String value){
          this.packetSize = Integer.parseInt(value);
        }

Next, int the prop file:

 PacketSize=2048 

Warning: the key in the prop file must be the same of the setter name without the prefix set.

Now, to load the prop file:

        // Level: Manager
        // At the execution load properties
        manager.execute(yes);
        // To load properties from differents  types of sources
        manager.loadAttributes();
        manager.loadAttributes(java.io.File propFile);
        manager.loadAttributes(java.util.Properties javaProp);
        // Level: Test
        
        // To load properties from differents  types of sources
        test.loadAttributes();
        test.loadAttributes(java.io.File propFile);
        test.loadAttributes(java.util.Properties javaProp);

<Manager>
  <name>A Manager </name>
  <description>Try XML descriptor file. </description>
  <!-- by default nbRuns is 1, but for benchmarks you can change it -->
  <nbRuns>100 </nbRuns>
</Manager>

<simpleGroup name="A simple Group" description="just for test.">
      <unitTest class="test.objectcreation.TestNewActive"/>
      <unitTest class="test.groupmigration.TestGroupCreation"/>
      <unitTest class="test.groupmigration.TestGroupCreation"/>
      <unitTest class="test.objectcreation.TestNewActive"/>
 </simpleGroup>

You have created a group with 4 tests.

<packageGroup name="A Package Group" 
              description="Construct Group from package." 
              dir="/net/home/adicosta/workspace/ProActive/classes" 
              packageName="nonregressiontest" >
</packageGroup>

You have created a group with all Tests was found in the package nonregressiontest

With this method you don't have any order on Tests, but you can specify some order:

<packageGroup name="A Package Group" description="Construct Group from  package." 
                dir="/net/home/adicosta/workspace/ProActive/classes" 
                packageName="nonregressiontest" >
     <unitTest class="nonregressiontest.runtime.defaultruntime.Test" />
     <unitTest class="nonregressiontest.node.nodefactory.Test" />
     <unitTest class="nonregressiontest.stub.stubgeneration.Test" />
     <unitTest class="nonregressiontest.stub.stubinterface.Test" />
     <unitTest class="nonregressiontest.activeobject.creation.local.newactive.Test" />
     <unitTest class="nonregressiontest.activeobject.creation.local.turnactive.Test" />
     <unitTest class="nonregressiontest.activeobject.creation.remote.newactive.Test" />
     <unitTest class="nonregressiontest.activeobject.creation.remote.turnactive.Test" />
</packageGroup>

All classes in package nonregressiontest are added, only the specified tests are sorted.

<interLinkedGroup name="Group with interlinked tests" description="Construct a Group with
 interlinked tests">
         <!-- Declare the tests in the execution order -->
 <idTest class="test.groupmigration.TestGroupCreation" id="1"/>
 <idTest class="test.groupmigration.TestGroupMigration" id="2"/>
 <idTest class="test.groupmigration.TestGroupMessage" id="3"/>
         <interLinks>
                 <link id="3">
                   <parent id="1"/>
                   <parent id="2"/>
                 </link>
           </interLinks>
 </interLinkedGroup>

TestGroupMessage depends from TestGroupCreation and TestGroupMigration.

<log4j>
  /net/home/adicosta/log4j/config/file/path/log4j-file-config
</log4j>

Results in a text file:

 <result type="text" file="/net/home/adicosta/tmp/results.txt" /> 

Results in a HTML file:

 <result type="html" file="/net/home/adicosta/tmp/results.html" /> 

Results in the console:

 <result type="console" /> 

Results in a XML file:

 <result type="xml" file="/net/home/adicosta/tmp/results.xml"/> 

To execute all with the XML file configuration:

Manager manager = new Manager(java.io.File xmlConfigFile);
manager.execute();

Like in simple prop file:

<properties>
   <prop key="RemoteHostname" value="nahuel"/>
</properties>

In this example both methods are empty, but they could be overridden in order to initialize and finalyze the test.

We will simply verify if the node is created:

    public boolean preConditions() throws Exception {
        return getNode() != null;
    }

This method is the test, we will create an active object:

 private ReifiableObject active = null;
  public void action() throws Exception {
        active = (ReifiableObject) ProActive.newActive(ReifiableObject.class.getName(),
                null, getNode());
    }

Remarks: The ReifiableObject class is a simple class who just extends java.lang.Object, implements java.io.Serilizable and has an empty constructor with no argument.

We will check if active is different of null and if the node contains active:

 public boolean postConditions() throws Exception {
        Object[] activeObjects = getNode().getActiveObjects();
        return (active != null) && (activeObjects != null) &&
        (activeObjects.length == 1) && activeObjects[0].equals(active);
    }

import org.objectweb.proactive.ProActive;
import org.objectweb.proactive.core.node.Node;
import testsuite.test.ProActiveFunctionalTest;
public class TestNewActive extends ProActiveFunctionalTest {
    private ReifiableObject active = null;
    public TestNewActive() {
        super();
        setName('newActive');
        setDescription('Test object creation with newActive in a node.');
    }
    public TestNewActive(Node node, String name) {
        super(node, name,
            'Test object creation with newActive in a node.');
    }
    public void initTest() throws Exception {
        // nothing to do
    }
    public boolean preConditions() throws Exception {
        return getNode() != null;
    }
    public void action() throws Exception {
        active = (ReifiableObject) ProActive.newActive(ReifiableObject.class.getName(),
                null, getNode());
    }
    public boolean postConditions() throws Exception {
        Object[] activeObjects = getNode().getActiveObjects();
        return (active != null) && (activeObjects != null) &&
        (activeObjects.length == 1) && activeObjects[0].equals(active);
    }
    public void endTest() throws Exception {
           // nothing to do
    }
}

Tips: if you want to make a trace in your test or in all classes who extends a testsuite class, you have access to a log4j logger by: getLogger()

Normaly, you have nothing to do to initialize the manager. In this example, we choose to create tests and group in this method , but you can do this in the same place where you create the manager.

Create group by the initManager():

import testsuite.group.Group;
public void initManager() throws Exception {
        Group testGroup = new Group('Test Group', 'no description.');
        // adding a test in same VM
        testGroup.add(new TestNewActive(getSameVMNode(),'NewActive same VM'));
        // adding a test in local VM
        testGroup.add(new TestNewActive(getLocalVMNode(),'NewActive local VM'));
        // adding a test in remote VM
        testGroup.add(new TestNewActive(getRemoteVMNode(),'NewActive remote VM'));
        // adding the group
        add(testGroup);
    }

Create group in the same place of the manager:

ObjectCreationManager manager = new ObjectCreationManager();
Group testGroup = new Group('Test Group', 'no description.');
// adding a test in same VM
testGroup.add(new TestNewActive(getSameVMNode(),'NewActive same VM'));
// adding a test in local VM
testGroup.add(new TestNewActive(getLocalVMNode(),'NewActive local VM'));
// adding a test in remote VM
testGroup.add(new TestNewActive(getRemoteVMNode(),'NewActive remote VM'));
// adding the group
manager.add(testGroup);

Warning: if you override endManager() method in a ProActiveManager you must to add in this code:

 super.endManager() 

The reason is to delete the ProActive nodes create at the beginning.

Our manager is a ProActiveManager, so an attibutes file is mandatory.

Create a file ObjectCreationManager.prop in the same directory of the manager. This file must contains the name (or URL) of the remote host, like this:

 RemoteHostname=owenii  

Warning: respect the upper an lower cases.

Tips: you can use this file to specify attributes for your tests classes. You can also use a different file, in this case you must specify its path in the execute() method of the manager.

8/22/03 13:48:10.450 [MESSAGE] Local hostname: amda.inria.fr
8/22/03 13:48:10.450 [MESSAGE] Remote hostname: owenii
8/22/03 13:48:10.452 [MESSAGE] Starting ...
8/22/03 13:48:10.458 [MESSAGE] Init Manager with success
8/22/03 13:48:10.749 [RESULT] NewActive same VM: Test run with success [SUCCESS]
8/22/03 13:48:11.141 [RESULT] NewActive local VM: Test run with success [SUCCESS]
8/22/03 13:48:12.195 [RESULT] NewActive remote VM: Test run with success [SUCCESS]
8/22/03 13:48:12.195 [RESULT] Group: Test Group Runs: 3 Errors: 0 [SUCCESS]
8/22/03 13:48:12.195 [MESSAGE] ... Finish

In this exampple both methods are empty, but they could be overridden in order to initialize and finalyze the benchmark.

We will simply verify if the node is created:

   public boolean preConditions() throws Exception {
     return getNode() != null;
   }

This method measures the time of a creation of an Object with ProActive.newActive() on a specified node:

    private ReifiableObject object = null;
    public long action() throws Exception {
        ReifiableObject object;
        String className = ReifiableObject.class.getName();
        Node node = getNode();
        this.timer.start();
        object = (ReifiableObject) ProActive.newActive(className, null, node);
        this.timer.stop();
        return this.timer.getCumulatedTime();
    }

	Note
	It is the benchmark's programmer who measure the time of the action with a configurable timer, see the Section 48.2, “Timer for the Benchmarks” for more details.

The ReifiableObject class is a simple class who just extends java.lang.Object, implements java.io.Serilizable and has an empty constructor with no argument.

We will check if object is different of null and if the node contains object:

 public boolean postConditions() throws Exception {
     Object[] activeObjects = getNode().getActiveObjects();
     return (object != null) && (activeObjects != null) &&
       (activeObjects.length == 1) && activeObjects[0].equals(object);
   }

Tips: if you want to make a trace in your benchmark , you have access to a log4j logger by: getLogger() or by the variable logger

Normaly, you have nothing to do to initialize the manager. In this example, we choose to create benchmarks and group in this method , but you can do this in the same place where you create the manager.

Create group by initManager():

import testsuite.group.Group;
  public void initManager() throwsException {
    Group benchGroup = new Group('Bnechmark Group','no description.');
    // adding bench in same VM
    benchGroup.add(new BenchNewActive(getSameVMNode()));
    // adding bench in local VM
    benchGroup.add(new BenchNewActive(getLocalVMNode()));
    // adding bench in remote VM
    benchGroup.add(new BenchNewActive(getRemoteVMNode()));
    // adding the group
    add(benchGroup);
  }

Create group int the same place of the manager:

  // ...
  Manager manager  = new Manager();
  Group  benchGroup = new Group('Bnechmark Group','no description.');
  // adding bench in same VM
  benchGroup.add(new BenchNewActive(getSameVMNode()));
  // adding bench in local VM
  benchGroup.add(new BenchNewActive(getLocalVMNode()));
  // adding bench in remote VM
  benchGroup.add(new BenchNewActive(getRemoteVMNode()));
  manager.add(benchGroup);
  // ...

Warning: if you override endManager() method in a ProActiveManager you must to add in this code:

 super.endManager() 

The reason is to delete the ProActive nodes create at the beginning.

Our manager is a ProActiveManager, so an attibutes file is mandatory.

Create a file Manager.prop in the same directory of the manager. This file must contains the name (or URL) of the remote host, like this:

 RemoteHostname=owenii 

Warning: respect the upper an lower cases.

Tips: you can use this file to specify attributes for your tests classes. You can also use a different file, in this case you must specify its path in the execute() method of the manager.

Results in your console:

 System.out.println(manager); 

If you want all details:

 manager.setVerbatim(true); 

For benchmarks it is more interesting to export results in a HTML file. Indeed, you have average, min, max, STDEV and charts to help you to analyse all results

Object Creation
Object Creation with newActive and turnActive. 
Messages of Object Creation:
9/18/2003 at 13:0:32.527 [RESULT] 
Object Creation with newActive -- Same VM: no message [SUCCESS] See the chart
Max=113ms Moy=24.0ms STDEV=24.64ms --> Min1ms
9/18/2003 at 13:0:36.693 [RESULT]
 Object Creation with turnActive -- Same VM: no message [SUCCESS]See the chart
 Max=98ms Moy=41.0ms STDEV=32.20ms --> Min1ms
9/18/2003 at 13:0:43.425 [RESULT]
 Object Creation with newActive -- Local VM: no message [SUCCESS]See the chart
 Max=376ms Moy=67.03ms STDEV=83.73ms --> Min6ms
9/18/2003 at 13:0:50.434 [RESULT]
 Object Creation with turnActive -- Local VM: no message [SUCCESS]See the chart
 Max=326ms Moy=69.82ms STDEV=86.15ms --> Min6ms
9/18/2003 at 13:0:53.297 [RESULT]
 Object Creation with newActive -- Remote VM: no message [SUCCESS]See the chart
 Max=290ms Moy=28.03ms STDEV=50.79ms --> Min5ms
9/18/2003 at 13:0:55.980 [RESULT]
 Object Creation with turnActive -- Remote VM: no message [SUCCESS]See the chart
 Max=250ms Moy=26.32ms STDEV=53.46ms --> Min5ms
9/18/2003 at 13:0:55.982 [RESULT]:
 Group: Object Creation, Moy in 42.7ms Runs: 600 Errors: 0
 To see all results of this group in a BarChart.

Create a new class who extends testsuite.test.ProActiveFunctionalTest, it is an abstract class.

See this template code:

import org.objectweb.proactive.core.node.Node;
import testsuite.test.ProActiveFunctionalTest;
import java.io.Serializable;
public class TestGroupCreation extends ProActiveFunctionalTest
    implements Serializable {
    public TestGroupCreation() {
        super(null, 'Group Creation',
            'Create a Group of active object in specify node.');
    }
    public TestGroupCreation(Node node) {
        super(node, 'Group Creation',
            'Create a Group of active object in specify node.');
    }
    public void action() throws Exception {
    }
    public boolean postConditions() throws Exception {
    }
    public boolean preConditions() throws Exception {
    }
    public void initTest() throws Exception {
        // nothing to do
    }
    public void endTest() throws Exception {
        // nothing to do
    }
}

Next we will simply test in preconditions if the node exists (different of null):

   public boolean preConditions() throws Exception {
        return getNode() != null;
    }

Now we will implement the action method to create a ProActive Group with 3 Agent (see the Agent code at the end of this section - Example 48.2, “Agent class”):

import org.objectweb.proactive.core.group.Group;
import org.objectweb.proactive.core.group.ProActiveGroup;
public class TestGroupCreation extends ProActiveFunctionalTest
    implements Serializable {
     private Agent group = null;
 
    // ...
  public void action() throws Exception {
             createGroupAgent();
 }
   
    private void createGroupAgent() throws Exception {
          Object[][] params = {
                       { 'Agent0' },
                       { 'Agent1' },
                       { 'Agent2' }
                };
          Node node = getNode();
              Node[] nodes = { node };
            group = (Agent) ProActiveGroup.newGroup(Agent.class.getName(), params,
              nodes);
     }
       // ...
}

Remarks: We use an external method to create the group is for the simple reason of we use this code after in another method.

Remarks: We don't explain the Agent code because it is a ProActive example.

For the postconditions we will test if the group containts 3 elements and they are in the good node:

    public boolean postConditions() throws Exception {
        if (group == null) {
            return false;
        } else {
            Group agentGroup = ProActiveGroup.getGroup(group);
            if (agentGroup.size() != 3) {
                return false;
            } else {
                Agent agent0 = (Agent) agentGroup.get(0);
                Agent agent1 = (Agent) agentGroup.get(1);
                Agent agent2 = (Agent) agentGroup.get(2);
                String nodeURL = getNode().getNodeInformation().getURL()
                                     .toUpperCase();
                return (agent0.getNodeName().compareTo(nodeURL) == 0) &&
                (agent1.getNodeName().compareTo(nodeURL) == 0) &&
                (agent2.getNodeName().compareTo(nodeURL) == 0);
            }
        }
    }

This class is now readi for a standalone use.

Now, we will add a new action method who return a ProActive Group:

    public Agent action(Object o) throws Exception {
        createGroupAgent();
        return this.group;
    }

This method return an Agent (who is the group) and have one argument: o. This argument will not use , we must to put this argument is for have a different method signature from action().

Our test for group creation is now ready.

In this test we can't run this method in a standalone test, but for other maybe you can. It is just for this test.

    public void action() throws Exception {
        throw new Exception('This test doesn't work in standalone mode');
    }

The result of the precedent test is an Agent, so the argument will be an Agent. This test have no result but we must to return an Object here it is null because the API use the reflection mechanism of Java.

    public Object action(Agent group) throws Exception {
        this.group = group;
        this.group.moveTo(getNode().getNodeInformation().getURL());
        return null;
    }

8/26/03 12:40:47.407 [MESSAGE] Local hostname: amda.inria.fr
8/26/03 12:40:47.408 [MESSAGE] Remote hostname: owenii
8/26/03 12:40:47.498 [MESSAGE] Starting with interlinked Tests ...
8/26/03 12:40:47.498 [MESSAGE] Init Manager with success
8/26/03 12:40:48.547 [RESULT] Group Creation: Test run with success [SUCCESS]
8/26/03 12:40:50.149 [RESULT] Group Migration: Test run with success [SUCCESS]
8/26/03 12:40:50.149 [RESULT] Group: Group Migration Runs: 2 Errors: 0 [SUCCESS]
8/26/03 12:40:50.243 [MESSAGE] ... Finish