OOSPMD

15.1. OOSPMD: Introduction

SPMD stands for Single Program Multiple Data, which is a technique used in paralellizing applications by separating task and running them simultaneously on different machines or processors. ProActive allows the use of object oriented programming combined with the SPMD techniques.

ProActive uses group communication with SPMD in order to free the programmer from having to implement the complex communication code required for setting identical groups in each SPMD activity. Group communication allows the focus to be on the application itself and not on the synchronizations. An SPMD group is a group of active objects where each one has a group referencing all the active objects.

This chapter presents the mechanism of typed group communication as an new alternative to the old SPMD programming model. While being placed in an object-oriented context, this mechanism helps the definition and the coordination of distributed activities. The approach offers a better structuring flexibility and implementation through a modest size API. The automation of key communication mechanisms and synchronization simplifies code writing.

The typed group communication system can be used to simulate MPI-style collective communication. Contrary to MPI that requires all members of a group to collectively call the same communication primitive, our group communication scheme makes possible for one activity to call methods on the group.

15.2. SPMD Group Creation

The main class for the SPMD groups is org.objectweb.proactive.api.PASPMD. This class contains methods for creating and controlling ProActive SPMD groups.

Figure 15.1. 


An SPMD group is a ProActive typed group built with the PASPMD.newSPMDGroup() method. This method looks like the PAGroup.newGroup() method; they have similar behavior (and overloads). The difference is that each member of an SPMD group has a reference to a group containing all the others members and itself (i.e. a reference to the SPMD group itself).

For a standard Java class:

public class A implements InitActive, RunActive, EndActive, java.io.Serializable {

   /**
    *
    */
   private String name = "anonymous";
   private boolean onewayCallReceived = false;

   public A() {
   }

The SPMD group is built as follows:

Object[][] params = { { "Agent0" }, { "Agent1" }, { "Agent2" } };
Node[] nodes = { NodeFactory.getDefaultNode(), super.getANode(), super.getANode() };
this.spmdgroup = (A) PASPMD.newSPMDGroup(A.class.getName(), params, nodes);

Object members of an SPMD group are aware about the whole group. They can obtain some informations about the SPMD group they belong to such as the size of the group, their rank in the group, and a reference to the group in order to get more informations or to communicate with method invocations. Those informations are respectively obtained using the following static methods of the PASPMD class.

public static int getMySPMDGroupSize()
public static int getMyRank()

and

public static Object getSPMDGroup()

15.3. Synchronizing activities with barriers

Synchronizing processes is an important operation in any distributed system. Managing concurrency when several processes try to reach the same resource and coordinating the action of several processes to perform a joint computation are important parts of distributed computing

There are different techniques to synchronize processes. With a SPMD application, we have the synchronization barriers mechanism.

Traditional barriers as those provided for example by MPI have a blocking behavior. When a process meets such a barrier, it fully stops its execution immediately and informs all the other members of the SPMD group about its state. It is only when all the other members have reach this barrier too that all the processes will be able to continue their execution. As the barriers are often used to synchronize the communication between processes, it is inefficient to completely stop the execution of a process without distinction between local computation and communication.

As a solution to this inefficiency ProActive provides non-blocking barriers to only synchronize communication between activities. When an activity reaches a barrier instruction while executing a service, it will continue the execution of the current service until its end and tag all the next outgoing requests as post barrier. Consequently, any activity receiving a request tagged like this will delay its service until all the activities implied in this synchronization are ready to synchronize. An example of this behavior is illustrated on the Figure 15.2, “Behaviour example of a total barrier”.

This example presents the timelines of three active objects which are performing one type of barrier called a total barrier. The requests A and B are served before the synchronization as they were sent before the reach of the barrier instruction (symbolized by a red point). On the contrary, the requests X and Y are served after the synchronization, as they were sent after the barrier instruction.

Behaviour example of a total barrier

Figure 15.2. Behaviour example of a total barrier


The OOSPMD programming model provided by ProActive offers three kinds of barriers to synchronize activities:

  • the Total Barrier

  • the Neighbour Barrier

  • the Method-Based Barrier

15.3.1. Total Barrier

Total barrier directly involves the SPMD group. A call to

public static void totalBarrier(String barrierName)

will block until all the members in the SPMD group have reached and called the identical barrier primitive. Such a call communicates with all the members of the SPMD group. The barrier is released when the Active Object has received a barrier message from all other members of the SPMD group (including itself).

[Warning] Warning

The string parameter is used as a unique identity name for the barrier. It is the programmer responsibility to ensure that two (or more) different barriers with the same id name are not invoked simultaneously.

Let us take a Java class that contains a method calling a total barrier, here the method foo:

myspmdgroup.foo();
PASPMD.totalBarrier("'1'");
myspmdgroup.bar();
PASPMD.totalBarrier("'2'");
myspmdgroup.gee();

Note that usually, strings used as unique ID are more complex; they can be based on the full name of the class or the package (org.objectweb.proactive.ClassName), for example. The SPMD group is built as follow:

Object[][] params = { { "Agent0" }, { "Agent1" }, { "Agent2" } };
Node[] nodes = { NodeFactory.getDefaultNode(), super.getANode(), super.getANode() };
this.spmdgroup = (A) PASPMD.newSPMDGroup(A.class.getName(), params, nodes);

When the method start() is called on different instances of the class containing it, the different instances will wait for myspmdgroup.foo() to be completed and the barrier call to be made before they start myspmdgroup.bar(). The instances will also synchronize after the call to myspmdgroup.bar() as there is also a second barrier PASPMD.totalBarrier("'2'").

The programmer has to ensure that all the members of an SPMD group call the barrier method otherwise the members of the group may indefinitely wait.

15.3.2. Neighbour barrier

The Neighbour barrier is a kind of lightweight barrier, involving only the Active Objects specified in a given group.

neighbourBarrier(String,group) initiates a barrier only with the objects of the specified group. Those objects, that contribute to the end of the barrier state, are called neighbours as they are usually local to a given topology, an object that invokes the Neighbour barrier HAVE TO BE IN THE GROUP given as parameter. The barrier message is only sent to the group of neighbours.

The programmer has to explicitly build this group of neighbours. The topology API can be used to build such group. Topologies are groups. They just give special access to their members or (sub)groups members. For instance, a matrix fits well with the topology Plan that provides methods to get the reference of neighbour members (left, right, up, down). See the javadoc of the topology package for more information:

org.objectweb.proactive.core.group.topology

Like for the Total barrier, the string parameter represents a unique identity name for the barrier. The second parameter is the group of neighbours built by the programmer. Here is an example:

// synchronization to be sure that all submatrix have exchanged borders
PASPMD.neighbourBarrier("SynchronizationWithNeighbors" + this.iterationsToStop, this.neighbors);

Refer to the Jacobi example to see a use-case of the Neighbour barrier. Each submatrix needs only to be synchronized with the submatrixes which are in its cardinal neighbours.

This barrier increases the asynchronism and reduce the amount of exchanged messages.

15.3.3. Method Barrier

The Method barrier does no more involve extra messages to communicate (i.e. the barrier messages). Communications between objects to release a barrier are achieved by the standard method call and request reception of ProActive.

As a standard barrier, the method methodBarrier(String[]) will finish the current request served by the active object that calls it, but it then wait for a request on the specified methods to resume. The array of string contains the name of the awaited methods. The order of the methods does not matter. For example:

 PASPMD.methodBarrier({'foo', 'bar', 'gee'}); 

The caller will stop and wait for the three methods. bar or gee can came first, then foo. If one wants wait for foo, then wait for bar, then wait for gee, three calls can be successively done:

   PASPMD.methodBarrier({'foo'});
   PASPMD.methodBarrier({'bar'});
   PASPMD.methodBarrier({'gee'}); 

A method barrier is used without any group (SPMD or not). To learn more on Groups, please refer to Chapter 10, Typed Group Communication.

15.4. MPI to ProActive Summary

<title>MPI to ProActive</title>
MPI ProActive
MPI_Init and MPI_Finalize Activities creation
MPI_Comm_Size PASPMD.getMyGroupSize
MPI_Comm_Rank PASPMD.getMyRank
MPI_Send and MPI_Recv Method call
MPI_Barrier PASPMD.barrier
MPI_Bcast Method call on a group
MPI_Scatter Method call with a scatter group as parameter
MPI_Gather Result of a group communication
MPI_Reduce Programmer's method