P2P Infrastructure With ProActive

The P2P infrastructure is self-organized and configurable. When the infrastructure is running you have nothing to do to keep it up. There are 3 main parameters to configure:

All parameter descriptions and the way to change their default values are explained in Section 17.4.3, “ Peer-to-Peer properties ”. Next section shows how to configure the infrastructure when starting the P2P Service with the command line.

The bootstrapping or first contact problem is how a new peer can join the p2p infrastructure. We solved this problem by just specifying one or several addresses of supposed peers which are running in the p2p infrastructure. Next, we will explain how and where you can specify this list of peers.

Now, you just have to start peers. There are two ways to do so:

 startP2PService [-acq acquisitionMethod] [-port portNumber] [-s Peer ...] [-f PeersListFile]

45.1.1.2.  Usage Example

In this illustration, we will explain how to start a first peer and then how new peers can create a P2P network with the first one.

Start the first peer with rmi protocol and listening on port 2410:

first.peer.host$startP2PService.sh -acq rmi -port 2410

Now, start new peers and connect them to the first peer to create a tiny P2P network:

  second.peer.host$startP2PService.sh -acq rmi -port 2410 -s rmi://first.peer.host 
  third.peer.host$startP2PService.sh -acq rmi -port 2602 -s rmi://first.peer.host 

You could specify a different port number for each peer.

Use a file to specify the addresses of peers:

The file hosts.file:

 rmi://first.peer.host:2410
 rmi://third.peer.host:2602

 file.peer.host$startP2PService.sh -acq rmi -port 8989 -f hosts.file 

Lastly, a new peer joins the P2P network:

 last.peer.host$startP2PService.sh -acq rmi -port 6666 -s rmi://third.peer.host:2410 

Figure 45.1. Usage example P2P network (after firsts connections)

 $ ./build daemon 

 ProActive/p2p/src/common/proactivep2p.h 

 ProActive/p2p/build/proactivep2p 

 C:>ProActive\scripts\windows\p2p\Service\install.bat 

 C:>ProActive\scripts\windows\p2p\Service\remove.bat 

 <periods> 
  <period>
   <start day='monday' hour='18'
 minute='0'/>
   <end day='tuesday' hour='6'
 minute='0'/>
   </period>
   <period>
    <start day='saturday' hour='0'
 minute='0'/>
    <end day='monday' hour='6'
 minute='0'/>
   </period>
 </periods>
 <register>
  <registry url='trinidad.inria.fr'/>
  <registry url='amda.inria.fr'/>
  <registry url='tranquility.inria.fr'/>
  <registry url='psychoquack.inria.fr'/>
 </register>

 ProActive/p2p/build/logs/hostname 

You can customize some P2P settings such as:

With elements acq and port, if a P2P Service is already running with this configuration the descriptor will use this one, if not a new one is started.

In order to get nodes, the peerSet tag will allow you to specify entry point of your P2P Infrastructure.

You can get nodes from the P2P Infrastructure using the ProActive Deployment Descriptor as described above.

In fact you will ask for a certain number of nodes and ProActive will notify a 'listener' (one of your class), every time a new node is available.

 ProActiveDescriptor pad = PADeployment.getProactiveDescriptor('myP2PXmlDescriptor.xml');
 // getting virtual node 'p2pvn' defined in the ProActive Deployement Descriptor
 VirtualNode vn = pad.getVirtualNode('p2pvn');
 
 // adding 'this' or anyother class has a listener of the 'NodeCreationEvent'
 ((VirtualNodeImpl) vn).addNodeCreationEventListener(this);
 //activate that virtual node
 vn.activate();

As you can see, the class executing this code must implement an interface in order to be notified when a new node is available from the P2P infrastructure.

Basically you will have to implement the interface NodeCreationEventListener that can be found in package org.objectweb.proactive.core.event. For example, this method will be called every time a new host is acquired:

 public void nodeCreated(NodeCreationEvent event) 
 {
 // get the node
 Node newNode = event.getNode();
   // now you can create an active object on your node.
 }

You should carefully notice that you can be notified at any time, whatever the code you are executing, once you have activated the virtual node.

A short preview of a XML descriptor:

 <infrastructure>                        
  <services>
   <serviceDefinition id='p2pservice'>
    <P2PService nodesAsked='2' acq='rmi' 
             port='2410' NOA='10' TTU='60000'
             TTL='10'>
     <peerSet>
      <peer>rmi://localhost:3000</peer>
     </peerSet>
    </P2PService>
   </serviceDefinition>
  </services>
 </infrastructure>

A complete example of file is available, see Example C.8, “P2P configuration: sample_p2p.xml” .

The next figure shows a P2P Service started with a XML deployment descriptor (xml_path attribute). Six nodes are shared on different hosts:

Figure 45.2. A P2P Service which is sharing nodes deployed by a descriptor

The next little sample of code explains how, from an application, you can start a P2P Service and get nodes:

import org.objectweb.proactive.ProActive;
import org.objectweb.proactive.core.ProActiveException;
import org.objectweb.proactive.core.mop.ClassNotReifiableException;
import org.objectweb.proactive.core.node.Node;
import org.objectweb.proactive.core.node.NodeException;
import org.objectweb.proactive.core.node.NodeFactory;
import org.objectweb.proactive.core.runtime.ProActiveRuntime;
import org.objectweb.proactive.core.runtime.RuntimeFactory;
import org.objectweb.proactive.extra.p2p.service.P2PService;
import org.objectweb.proactive.extra.p2p.service.StartP2PService;
import org.objectweb.proactive.extra.p2p.service.node.P2PNodeLookup;
...
// This constructor uses a file with address of peers
// See the Javadoc to choose different parameters
StartP2PService startServiceP2P = new StartP2PService(p2pFile);
// Start the P2P Service on the local host
startServiceP2P.start();
// Get the reference on the P2P Service
P2PService serviceP2P = startServiceP2P.getP2PService();
// By the application's P2P Service ask to the P2P infrastructure
// for getting nodes.
P2PNodeLookup p2pNodeLookup = serviceP2P.getNodes(nNodes,
 virtualNodeName, JobID);
// You can migrate the P2P node lookup from the p2p service
// to an another node:
p2pNodeLookup.moveTo("//localhost/localNode");
// Use method from p2pNodeLookup to get nodes
// such as
while (! p2pNodeLookup.allArrived()) {
 Vector arrivedNodes = p2pNodeLookup.getAndRemoveNodes();
 // Do something with nodes
 ...
}
// Your application
...
// End of your program
// Free shared nodes
p2pNodeLookup.killAllNodes();

There are a lot of P2P definitions, many of them are similar to other distributed infrastructures, such as Grid, client / server, etc. There are 2 better definitions which describe really P2P well:

'[...] P2P is a class of applications that take advantage of resources - available at the edges of the Internet [...]'

'[...] Peers are accessible by other peers directly [...] Any arbitrary chosen peer can be removed from the network without fault [...]'

P2P's focus on sharing, decentralization, instability and fault tolerance.

The P2P infrastructure is implemented with ProActive. Thus the shared resource is not a JVMs but a ProActive node, nodes are like a container which receives work.

The P2P infrastructure is not directly implemented in the ProActive core at the ProActive runtime level because we choose to be above communication protocols, such as RMI, HTTP, Ibis, etc. Therefore, the P2P infrastructure can use RMI or HTTP as communication layer. Hence, the P2P infrastructure is implemented with classic ProActive active objects and especially with ProActive typed group for broadcasting communications between peers due to your inspired BFS.

Using active objects for the implementation is a good mapping with the idea of a peer which is an independent entities that works as a server with a FIFO request queue. The peer is also a client which sends requests to other peers.

The list of P2P active objects:

The Figure below shows the connection between all active objects:

Figure 45.7. Nodes and Active Objects which make up a P2P Service.

All communications between peers use Group communication but for sending a response to a request message, it's a point-to-point communication. Though ProActive communications are asynchronous, it's not really messages which are sent between peers. Nevertheless, it's not a real problem; ProActive is implemented above Java RMI which is RPC and RPC is synchronous. However, ProActive uses future mechanism and Rendez-vous method to turn RPC methods to asynchronous. That means ProActive is asynchronous RPC. Rendez-vous is interesting in your case because it guarantees the method is successfully received by the receiver. With the Heart beat message which is sent a Java exception when an acquaintance is down.

The P2PAcquaintanceManager manages the list of acquaintances, this list is represented by a ProActive typed group of P2PService. This is the point of the next section.

ProActive typed group does not allow access to group elements and make calls from different active objects to the same group is not possible, i.e. a group can not be shared. However, the point of the P2P infrastructure is to broadcast messages to all members on the acquaintance list, ProActive typed group is perfect for doing that. A typed group of P2PService is a good implementation of the acquaintance list design.

But a typed group does not support to be shared by many active objects, especially for making group method calls from different objects, adding / removing / etc. members in the group. For the P2P infrastructure the P2PAcquaintanceManager (PAM) was designed.

The PAM is a standard active object, at its initialization it constructs an empty P2PService group. The PAM provides an access to few group methods, such as removing, adding and group size methods. All other active objects, such as P2PService or P2PNodeLookup, have to use PAM methods to access the group. The PAM works as a server with an FIFO queue behind the group.

That solves the problem of group members accessing but not how other active objects can call methods on the group. The ProActive group API provides a method to active a group that is made possible to get ProActive reference on the group. The PAM actives the group after its creation. P2PService, P2PNodeLookup and all get the group reference from a PAM's getter.

The PAM, during its activity, frequently sends heart beats to remove unavailable peers. The P2PService adds, via the PAM, new discovered acquaintances (P2PService) and the P2PNodeLookup calls group methods to ask nodes to the group reference. The P2PService does also group method calls.

In short, this can be seen in the next Figure:

Figure 45.8. Dynamic Shared ProActive Typed Group.

We just explained how to share a typed group between active objects but that is not solve all the problems. For the moment, the BFS implementation with broadcasting to all acquaintances each time is not perfect due to the message which is always send back to the previous sender. We are working to add member exclusion in a group method call.

The sharing node mechanism is an independent activity from the P2P service. Nodes are the sharing resource of this P2P network. This activity is handled by the P2PNodeManager active object.

At the initialization of the P2PNodeManager (PNM), it has to instantiate the shared resource. By default, it's 1 ProActive nodes by CPUs, for example on a single processor machine the PNM starts 1 node and on a bi-processors machine it starts 2 nodes. It's possible to choose to share only a single node. An another way is to share nodes from an XML deployment descriptor file by specifing the descriptor to the PNM which actives the deployment and gets nodes ready to share.

When the P2P service receives a node request, the request is forwarded (after the BFS broadcast) to the PNM which checks for a free node. In the case of at least 1 free node, the PNM must book the node and send back a reference to the the node to the original request sender. However, the booking remains valid for a predetermined time, this time expires after a configurable timeout. The PNM knows if the node is used or not by testing the active object presence inside the node. Consequently, at the end of the booking time, the PNM kills the node, the node is no longer usable. Though, some applications need empty nodes for a long time before using them, thereby there is a pseudo expand booking time system: creating 'Dummy' active objects in booked nodes for later use. This system is allowed by the P2PNodeLookup.

The P2PNodeLookup could receive more nodes than it needs, for all additional nodes, the P2PNodeLookup sends a message to all PNMs' nodes to cancel its booking on the node.

The deployed applications have to leave nodes after use. Therefore, the PNM offers a leaving node mechanism that is the application sent a leaving message for a specified node to the PNM which kills all node's active objects by terminating their bodies and kills the node. After that, the PNM creates a new node which is ready for sharing. However, if nodes are deployed by an XML descriptor the PNM does't kill the node, it just terminates all its active objects and re-shares the same node.

The asking node mechanism is allowed by the P2PNodeLookup, this object is active by the P2PService when it receives an asking node request from an application. The P2PNodeLookup (PNL) works as a broker, it could migrate to another place (node, machine, etc.) to be near the application.

The PNL aims to find the number of nodes requested by the application. It uses the BFS to frequently flood the network until it gets all nodes or until the timeout is reached. However, the application can ask to the maximum number of nodes, in that case the PNL asks to nodes until the end of the application. The PNL provides a listener / producer event mechanism which is great for the application which wants to know when a node is found.

Finally, the application kills nodes by the PNL which is in charge of contacting all the PNMs of each node and asks them to leave nodes. The PNMs leave nodes with the same mechanism of the booking timeout.

Lastly, the asking nodes mechanism with the PNL is fully integrated to the ProActive XML deployment descriptor.

IC2D hides all P2P internal object by default, in order to monitor the infrastructure itself we invite you to read the ProActive Optimizing documentationto to set the right option.

A screen shot made with IC2D. You can see 3 P2P services which are sharing 2 nodes (bi-processors machines). Inside the nodes there are some active Domain objects from the nBody application which is deployed on this small P2P infrastructure.

Figure 45.9. nBody application deployed on P2P Infrastructure.