| Prev | Part IV. Deploying Chapter 21. XML Deployment Descriptors |  | Next |
Parameters tied to the deployment of an application should be totally described in a xml deployment descriptor. Hence within the source code, there are no longer any references to:
Machine names
Creation Protocols
Registry/Lookup and Communications Protocols
rmi
http
rmissh
ibis
soap
Files Transfers
scp, rcp
unicore, arc (nordugrid)
other protocols like globus, glite will be supported soon
A ProActive application can be deployed on different hosts, with different protocols without changing the source code
Within a ProActive program, active objects are still created on Nodes
newActive(String, Object[], Node);
Nodes can be obtained from VirtualNodes (VN) declared and defined in a ProActiveDescriptor
Nodes are actual entities:
running into a JVM, on a host
they are the result of mapping VN --> JVMs
After activation the names of Nodes mapped with a VirtualNode are VirtualNode name + random number
VNs have the following characteristics:
a VN is uniquely identified as a String ID
a VN is defined in a ProActiveDescriptor
a VN has an object representation in a program after activation
Additional methods are provided to create groups of active objects on VirtualNodes. In that case an Active Object(member of the group) is created on each nodes mapped to the VirtualNode given as parameter
newActiveAsGroup(String, Object[], VirtualNode); turnActiveAsGroup(Object, String, VirtualNode);
Within a ProActiveDescriptor file, it is specified:
the mapping of VN to JVMs
the way to create, acquire JVMs using processes defined in the lower infrastructure part
local, remote processes or combination of both to create remote jvms.
For instance defining an sshProcess that itself references a local jvmProcess. At execution, the ssh process will launch a jvm on the remote machine specified in hostname attribute of sshProcess definition.
files transfers
fault tolerance, P2P, security
Example:
<ProActiveDescriptor xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:noNamespaceSchemaLocation='DescriptorSchema.xsd'> <componentDefinition> <virtualNodesDefinition> <virtualNode name='Dispatcher'/> </virtualNodesDefinition> </componentDefinition> <deployment> <mapping> <map virtualNode='Dispatcher'> <jvmSet> <vmName value='Jvm1'/> </jvmSet> </map> </mapping> <jvms> <jvm name='Jvm1'> <creation> <processReference refid='jvmProcess'/> </creation> </jvm> </jvms> </deployment> <infrastructure> <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> </processes> </infrastructure> </ProActiveDescriptor>This example shows a VirtualNode called Dispatcher, that is mapped to a jvm called Jvm.
This Jvm1will be created using the process called jvmProcess which is defined in the infrastructure part(This part will be discussed later, just notice that there are two parts in the descriptor, an abstract one containing VirtualNode definition and deployment informations and a more concrete one containing concrete infrastructure informations, that is where all processes are defined).
Typical example of a program code:
ProActiveDescriptor pad = ProActive.getProactiveDescriptor(String xmlFile);
//--------- Returns a ProActiveDescriptor object from the xml file
VirtualNode dispatcher = pad.getVirtualNode('Dispatcher');
//-------- Returns the VirtualNode Dispatcher described in the xml file as a java object
dispatcher.activate();
// -------- Activates the VirtualNode
Node node = dispatcher.getNode();
// -------- Returns the first node available among nodes mapped to the VirtualNode
C3DDispatcher c3dDispatcher = newActive(
'org.objectweb.proactive.core.examples.c3d.C3DDispatcher',param, node);
..........................Set of methods are provided in org.objectweb.proactive.descriptor.ProActiveDescriptor to manipulate VirtualNodes, to activate several VirtualNodes at the same time .... and in org.objectweb.proactive.core.descriptors.VirtualNode to manipulate and get nodes associated to VirtualNodes.
Mapping one to one: 1 VN --> 1 JVM
<virtualNodesDefinition> <virtualNode name='Dispatcher'/> </virtualNodesDefinition> <deployment> <mapping> <map virtualNode='Dispatcher'> <jvmSet> <vmName value='Jvm0'/> </jvmSet> </map> </mapping>Another possibility for the one to one mapping is to map 1 VN to the jvm running the program. In that case the lookup protocol can be specified but is optionnal(default value is the property proactive.communication.protocol) as it is shown in the following:
<virtualNodesDefinition> <virtualNode name='Dispatcher'/> </virtualNodesDefinition> <deployment> <mapping> <map virtualNode='Dispatcher'> <jvmSet> <currentJvm protocol='rmi'/> or <currentJvm/> </jvmSet> </map> </mapping>
Since it is the current jvm, it has not to be redifined later in the descriptor. This will be shown in a complete example
Mapping one to n: 1 VN --> N JVMs
<virtualNodesDefinition> <virtualNode name='Renderer' property='multiple'/> </virtualNodesDefinition> <deployment> <mapping> <map virtualNode='Renderer'> <jvmSet> <currentJvm/> <vmName value='Jvm1'/> <vmName value='Jvm2'/> <vmName value='Jvm3'/> <vmName value='Jvm4'/> </jvmSet> </map> </mapping>Note that the property attribute is set to multiple if you want to map 1 VN to multiple JVMs, and then a set of JVMs is defined for the VirtualNode Renderer. Four values are possible for the property attribute: unique which means one to one mapping, unique_singleAO: one to one mapping and only one AO deployed on the corresponding node, multiple: one to N mapping, multiple_cyclic: one to N mapping in a cyclic manner. This property is not mandatory but an exception can be thrown in case of incompatibility. For instance property set to unique, and more than one jvm defined in the jvmSet tag. In case of property set to unique_singleAO method getUniqueAO() in class org.objectweb.proactive.core.descriptor.data.VirtualNode called on such VirtualNode returns the unique AO created
Three other attributes timeout, waitForTimeout, minNodeNumber can be set when defining a virtualNode
<virtualNodesDefinition> <virtualNode name='Dispatcher' timeout='200' waitForTimeout='true'/> <virtualNode name='Renderer' timeout='200' minNodeNumber='3'/> </virtualNodesDefinition>
The timeout attribute represents an amount of time(in milliseconds) to wait before accessing Nodes mapped on the VirtualNode. The waitForTimeout attribute is a boolean. If set to true, you will have to wait exaclty timeout seconds before accessing Nodes. If set to false, timeout represents the maximum amount of time to wait, it means that if all nodes are created before the timeout expires, you get access to the Nodes. Defaut value for waitForTimeout attribute is false. The minNodeNumber attribute defines the minimum number of nodes to be created before accessing the nodes. If not defined, access to the nodes will occur once the timeout expires, or the number of nodes expected are effectively created. Setting this attribute allows to redefine the number of nodes expected, we define it as the number of nodes needed for the VirtualNode to be suitable for the application. In the exammple above, once 3 nodes are created and mapped to the VirtualNode Renderer, this VirtualNode starts to give access to its nodes. Those options are very usefull when there is no idea about how many nodes will be mapped on the VirtualNode(which is often unususal). Those attributes are optional.
Mapping n to one: N VN --> 1 JVMs
<virtualNodesDefinition> <virtualNode name='Dispatcher' property='unique_singleAO'/> <virtualNode name='Renderer' property='multiple'/> </virtualNodesDefinition> <deployment> <mapping> <map virtualNode='Dispatcher'> <jvmSet> <vmName value='Jvm1'/> </jvmSet> </map> <map virtualNode='Renderer'> <jvmSet> <vmName value='Jvm1'/> <vmName value='Jvm2'/> <vmName value='Jvm3'/> <vmName value='Jvm4'/> </jvmSet> </map> </mapping>In this example both VirtualNodes Dispatcher and Renderer have a mapping with Jvm1, it means that at deployment time, both VirtualNodes will get nodes created in the same JVM. Here is the notion of co-allocation in a JVM.
VirtualNode registration
Descriptors provide the ability to register a VirtualNode in a registry such RMIRegistry, JINI Lookup, HTTP registry, IBIS/RMI Registry Service. Hence this VirtualNode will be accessible by another application as it is described in the VirtualNodes Acquisition section. The protocol(registry) to use can be specified in the descriptor, if not specified, the VirtualNode will register using the protocol specified in proactive.communication.protocol property.
<virtualNodesDefinition> <virtualNode name='Dispatcher' property='unique_singleAO'/> <virtualNodesDefinition/> <deployment> <register virtualNode='Dispatcher' protocol='rmi'/> or <register virtualNode='Dispatcher'/> <!--using this syntax, registers the VirtualNode with the protocol specified in proactive.communication.protocol property --> <mapping> <map virtualNode='Dispatcher'> <jvmSet> <vmName value='Jvm0'/> </jvmSet> </map> </mapping>
The register tag allows to register the VirtualNode Dispatcher when activated, on the local machine in the RMIRegistry. As said before this VirtualNode will be accessible by another application using the lookup tag(see below) or using method: ProActive.lookupVirtualNode(String).
Descriptors provide the ability to acquire a VirtualNode already deployed by another application. Such VirtualNodes are defined in VirtualNodes Acquisition tag as it is done for VirtualNodesDefinition except that no property and no mapping with jvms are defined since such VNs are already deployed. In the deployment part, the lookup tag gives information on where and how to acquire the VirtualNode. Lookup will be performed when activating the VirtualNode.
<virtualNodesAcquisition> <virtualNode name='Dispatcher'/> </virtualNodesAcquisition> .......... <deployment> .......... <lookup virtualNode='Dispatcher' host='machine_name' protocol='rmi' port='2020'/> </deployment>
As mentioned in the previous section, in order to acquire VirtualNode Dispatcher, it must have been previously registered on the specified host by another application. Sometimes, the host where to perform the lookup will only be known at runtime, it that case it is specified in the descriptor with '*' for the host attribute
<lookup virtualNode='Dispatcher' host='*' protocol='rmi'/>
Then when the host name is available, ProActive provides method setRuntimeInformations in class org.objectweb.proactive.core.descriptor.data.VirtualNode to update the value and to perform the lookup. Typical example of code:
ProActiveDescriptor pad = ProActive.getProactiveDescriptor(String xmlFileLocation);
//----------- Returns a ProActiveDescriptor object from the xml file
pad.activateMappings;
// -------------------activate all VirtualNodes(definition and acquisition)
vnDispatcher =
pad.getVirtualNode('Dispatcher');
..........................
vnDispatcher.setRuntimeInformations('LOOKUP_HOST','machine_name);
//--------------set the property 'LOOKUP_HOST at runtime
To summarize all VirtualNodes are activated by calling activate methods except if '*' is set for a VirtualNode to be acquired. In that case the lookup will be performed when giving host informations.
Registration and lookup can be performed automatically when using tags in the descriptor as well as programmatically using static methods provided in org.objectweb.Proactive class:
ProActive.registerVirtualNode(
VirtualNode virtualNode,
String registrationProtocol,
boolean replacePreviousBinding );
ProActive.lookupVirtualNode(String url, String protocol);
ProActive.unregisterVirtualNode(VirtualNode virtualNode);
1 JVM --> 1 Node
........................... <jvm name='jvm1'> <creation> <processReference refid='jvmProcess'/> </creation> </jvm> .................................In this example, jvm1 will be created using the process called jvmProcess (discussed later, this process represents a java process and can be seen as java ProActiveClassname command)
1 JVM --> N Nodes on a single JVM
........................... <jvm name='jvm1' askedNodes='3'> <creation> <processReference refid='jvmProcess'/> </creation> </jvm> .................................
1 JVM --> N Nodes on N JVMs
This is the case when the process referenced is a cluster process(LSF, PBS, GLOBUS, ....) or a process list (see Process list)
Descriptors give the ability to acquire JVMs instead of creating them. To do so, it must be specified in the acquisition tag which service to use in oder to acquire the JVMs. Services will be described below, in the infrastructure part. At this point 2 services are provided: RMIRegistryLookup and P2PService service.
........................... <jvm name='jvm1'> <acquisition> <serviceReference refid='lookup'/> </acquisition> </jvm> .................................
In this example, Jvm1 will be acquired using the service called lookup (discussed later, this service represents a way to acquire a JVM). Note that the name lookup is totally abstract, with the condition that a service with the id lookup is defined in the infrastructure part
To avoid mistake when building XML descriptors, ProActive provides an XML Schema called DescriptorSchema.xsd. Then to validate your file against this schema, the following line must be put at the top of the xml document as it is done for all ProActive examples.
<ProActiveDescriptor
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:noNamespaceSchemaLocation='Location_of_DescriptorSchema.xsd'>
Note that this schema is available in the ProActive distribution package under ProActive\descriptor directory. Using descriptors related methods (Proactive.getProactiveDescriptor(file)) triggers automatic and transparent validation of the file using Xerces2_4_0 if the ProActive property schema.validation is set to enable (see Chapter 20, ProActive Basic Configuration for more details). If a problem occurs during the validation, an error message is displayed. Otherwise, if the validation is successful, no message appear. An XML validation tool such as XMLSPY5.0(windows) can also be used to validate XML descriptors.
Following XML files examples are used for the C3D application. The first file is read when launching the C3DDispatcher. The second one is read every time a C3DUser is added. Both files contain many features described earlier in this document.
<ProActiveDescriptor
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:noNamespaceSchemaLocation='DescriptorSchema.xsd'>
<componentDefinition>
<virtualNodesDefinition>
<virtualNode name='Dispatcher' property='unique_singleAO'/>
<virtualNode name='Renderer' property='multiple'/>
</virtualNodesDefinition>
</componentDefinition>
<deployment>
<register virtualNode='Dispatcher'/>
<mapping>
<map virtualNode='Dispatcher'>
<jvmSet>
<currentJvm/>
</jvmSet>
</map>
<map virtualNode='Renderer'>
<jvmSet>
<vmName value='Jvm1'/>
<vmName value='Jvm2'/>
<vmName value='Jvm3'/>
<vmName value='Jvm4'/>
</jvmSet>
</map>
</mapping>
<jvms>
<jvm name='Jvm1'>
<creation>
<processReference
refid='jvmProcess'/>
</creation>
</jvm>
<jvm name='Jvm2'>
<creation>
<processReference
refid='jvmProcess'/>
</creation>
</jvm>
<jvm name='Jvm3'>
<creation>
<processReference
refid='jvmProcess'/>
</creation>
</jvm>
<jvm name='Jvm4'>
<creation>
<processReference
refid='jvmProcess'/>
</creation>
</jvm>
</jvms>
</deployment>
<infrastructure>
<processes>
<processDefinition
id='jvmProcess'>
<jvmProcess
class='org.objectweb.proactive.core.process.JVMNodeProcess'/>
</processDefinition>
</processes>
</infrastructure>
</ProActiveDescriptor>
Example 21.1. C3D_Dispatcher_Render.xml
This example represents xml deployment descriptor for the C3D application. The abstract part containing VirtualNodes definition and deployment informations has already been explained. To summarize, two VirtualNodes are defined Dispatcher and Renderer. Dispatcher is mapped to the jvm running the main(), and will be exported using the protocol specified in proactive.communication.protocol property. This VirtualNode will be registered in a Registry(still using the protocol specified in proactive.communication.protocol property) when activated. Renderer is mapped to a set of JVMs called Jvm1, ..., Jvm4.
<ProActiveDescriptor
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xsi:noNamespaceSchemaLocation='DescriptorSchema.xsd'>
<componentDefinition>
<virtualNodesDefinition>
<virtualNode name='User'/>
</virtualNodesDefinition>
<virtualNodesAcquisition>
<virtualNode name='Dispatcher'/>
</virtualNodesAcquisition>
</componentDefinition>
<deployment>
<mapping>
<map virtualNode='User'>
<jvmSet>
<currentJvm/>
</jvmSet>
</map>
</mapping>
<lookup virtualNode='Dispatcher'
host='*' protocol='rmi'/>
</deployment>
<infrastructure>
<processes>
<processDefinition
id='jvmProcess'>
<jvmProcess
class='org.objectweb.proactive.core.process.JVMNodeProcess'/>
</processDefinition>
</processes>
</infrastructure>
</ProActiveDescriptor>
Example 21.2. C3D_User.xml
This file is read when addind a C3DUser. Two VirtualNodes are defined User which is mapped to the jvm running the main(), whose acquisition method is performed by looking up the RMIRegistry and Dispatcher in the virtualNodesAcquisition part which will be the result of a lookup in the RMIRegistry of a host to be specified at runtime.
In the previous example, all defined JVMs will be created using jvmProcess process. This name is abstract like the other ones, it means that it can be changed. This process is totally defined in the infrastructure part. Of course the process name in the creation part must point at an existing defined process in the infrastructure part. For instance if the name in the creation tag is localJVM, there must be a process defined in the infrastructure with the id localJVM
As said before, all processes are defined in the infrastructure part, under the processes tag. In the previous example, the
defined process jvmProcess will create
local JVMs. The class attribute defines the class to instantiate in
order to create the process. ProActive library provides one class to
instantiate in order to create processes that will launch local JVMs:
org.objectweb.proactive.core.process.JVMNodeProcess
<infrastructure> <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcesss'> <classpath> <absolutePath value='/home/ProActive/classes'/> <absolutePath value='/home/ProActive/lib/bcel.jar'/> <absolutePath value='/home/ProActive/lib/asm.jar'/> <absolutePath value='/home/ProActive/lib/jini-core.jar'/> <absolutePath value='/home/ProActive/lib/jini-ext.jar'/> <absolutePath value='/home/ProActive/lib/reggie.jar'/> </classpath> <javaPath> <absolutePath value='/usr/local/jdk1.4.0/bin/java'/> </javaPath> <policyFile> <absolutePath value='/home/ProActive/scripts/proactive.java.policy'/> </policyFile> <log4jpropertiesFile> <relativePath origin='user.home' value='ProActive/scripts/proactive-log4j'/> </log4jpropertiesFile> <ProActiveUserPropertiesFile> <absolutePath value='/home/config.xml'/> </ProActiveUserPropertiesFile> <jvmParameters> <parameter value='-Djava.library.path=/home1/fabrice/workProActive/ProActive/lib'/> <parameter value='-Dsun.boot.library.path=/home1/fabrice/workProActive/ProActive/lib'/> <parameter value=-Xms512 -Xmx512'/> </jvmParameters> </jvmProcess> </processDefinition> </processes> </infrastructure>
As shown in the example above, ProActive provides the ability to define or change the classpath environment variable, the java path, the policy file path, the log4j properties file path, the ProActive properties file path (see Chapter 20, ProActive Basic Configuration for more details) and also to pass parameters to the JVM to be created. Note that parameters to be passed here are related to the jvm in opposition to properties given in the configuration file (see Chapter 20, ProActive Basic Configuration), which is more focused on ProActive or application behaviour. In fact parameters given here will be part of the java command to create other jvms, whereas properties given in the config file will be loaded once the jvm is created.
If not specified, there is a default value (except for the jvmParameters element) for each of these variables. In the first example of this section, just the Id of the process, and the class to instantiate are defined. You might want to define the classpath or java path or policyfile path, etc, when creating remote JVMs(discussed later) if the home directory is not the same on your machine and on the machine where you want to create the JVM or for instance if you want to interact with Windows OS if you work on Linux and vice versa. As shown in the example paths to files can be either absolute or relative. If relative, an origin must be provided, it can be user.home or user.dir or user.classpath and it is resolved locally, i.e on the jvm reading the descriptor and not on the remote jvm that is going to be created.
As mentionned in the configuration file (see Chapter 20, ProActive Basic Configuration), if the <ProActiveUserPropertiesFile> is not defined for remote jvms, they will load a default one once created.
Even if not shown in this example, a specific tag is provided for XbootClasspath option under the form
<bootclasspath> <relativePath origin='user.home' value='/IOFAb/Ibis/'/> <relativePath origin='user.home' value='/IOFAb/classlibs/jdk'/> </bootclasspath>
With XML Deployment Descriptor, ProActive provides the ability to create remote Nodes (remote JVMs). You can specify in the descriptor if you want to access the remote host with rsh, ssh, rlogin, lsf, pbs, oar, prun, globus, unicore, arc (nordugrid), glite. How to use these protocols is explained in the following examples. Just remind that you can also combine these protocols.The principle of combination is fairly simple, you can imagine for instance that you will log on a remote cluster frontend with ssh, then use pbs to book nodes and to create jvms on each. You will also notice that there is at least one combination for each remote protocol. Indeed each remote protocol must have a pointer either on another remote protocol or on a jvmProcess to create a jvm(discussed previoulsy).
You can find here several examples of supported protocols and useful combinations.
Note that it is mandatory for using all these features, that ProActive is installed on each host, of course on the local host as well as on each host where you want to create Nodes
RSH
........................... <jvm name='jvm1'> <creation> <processReference refid='rshProcess'/> </creation> </jvm> ................................. <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='rshProcess'> <rshProcess class='org.objectweb.proactive.core.process.rsh.RSHProcess' hostname='sea.inria.fr'> <processReference refid='jvmProcess'/> </rshProcess> </processDefinition> </processes>For the Jvm2 the creation process is rshProcess(still an abstract name), which is defined in the infrastructure section. To define this process you have to give the class to instantiate to create the rsh process. ProActive provides
org.objectweb.proactive.core.process.rsh.RSHProcess
to create rsh process. You must
give the remote host name to log on with rsh. You can define as well
username='toto' if you plan to use rsh with
-l option. As said before this
rsh process must reference a local process, and in the
example, it references the process defined with the id jvmProcess. It means that once logged on
sea.inria.fr with rsh, a local JVM will be launched, ie a ProActive
node will be created on sea.inria.fr thanks to the process defined
by jvmProcess.Check examples/RSH_Example.xml for a complete rsh deployment example.
RLOGIN
........................... <jvm name='jvm1'> <creation> <processReference refid='rloginProcess'/> </creation> </jvm> ................................. <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='rloginProcess'> <rloginProcess class='org.objectweb.proactive.core.process.rlogin.RLoginProcess' hostname='sea.inria.fr'> <processReference refid='jvmProcess'/> </rloginProcess> </processDefinition> </processes>You can use rlogin in the same way that you would use rsh
SSH
........................... <jvm name='jvm1'> <creation> <processReference refid='sshProcess'/> </creation> </jvm> ................................. <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='sshProcess'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='sea.inria.fr'> <processReference refid='jvmProcess'/> </sshProcess> </processDefinition> </processes>ProActive provides
org.objectweb.proactive.core.process.ssh.SSHProcess
to create ssh process.In order to use ssh to log on a remote host, you must perform some actions. First you need to copy your public key (located in identity.pub under ~/.ssh on your local machine) in the authorized_keys(located under ~/.ssh) file of the remote host. Then to avoid interactivity, you will have to launch on the local host the ssh-agent command: ssh-agent $SHELL, this command can be put in your .xsession file, in order to run it automatically when logging on your station. Then launching ssh-add command to add your identity. Running this command will ask you to enter your passphrase, it is the one you provided when asking for an ssh key pair.
Note also that if the generated key pair is not encrypted (no passphrase), you do not need to run neither the ssh-agent, nor the ssh-add command. Indeed it is sufficient when using non encrypted private key, to only copy the public key on the remote host (as mentionned above) in order to get logged automatically on the remote host.
These steps must be performed before running any ProActive application using ssh protocol. If you are not familiar with ssh, see openSSH
Check examples/SSH_Example.xml for a complete ssh deployment example.
ProActive provides a way to define a list of processes for RSH, SSH, RLOGIN protocols. Using processList or processListbyHost elements avoids having a long deployment file when many machines with similar names are going to be connected with protocols mentionned before. The first example below shows how to use processList tag, the second how to use processListbyHost.
........................... <jvm name='jvm1'> <creation> <processReference refid='processlist'/> </creation> </jvm> ................................. <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='processlist'> <processList class='org.objectweb.proactive.core.process.ssh.SSHProcessList' fixedName='node-' list='[0-100;2]^[10,20]' padding='3' domain='sophia.grid5000.fr'> <processReference refid='jvmProcess'/> </processList> </processDefinition> </processes>
When using processList tag, the class attribute can take 3 values:
org.objectweb.proactive.core.process.ssh.SSHProcessList (see core/process/ssh/SSHProcessList.java),
org.objectweb.proactive.core.process.rsh.RSHProcessList (see core/process/rsh/RSHProcessList.java),
org.objectweb.proactive.core.process.rlogin.RLoginProcessList (see core/process/rlogin/RLoginProcessList.java),
........................... <jvm name='jvm1'> <creation> <processReference refid='processlist'/> </creation> </jvm> ................................. <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='processlist'> <processListbyHost class='org.objectweb.proactive.core.process.ssh.SSHProcessList' hostlist='crusoe waha nahuel' domain='inria.fr'> <processReference refid='jvmProcess'/> </processListbyHost> </processDefinition> </processes>
Using processListbyHost element allows to give a hostlist separated with a whitespace. The class attribute is defined as described in the processList tag. The domain attribute is optional since the complete hostname can also be provided in the hostlist attribute. In the example, a jvm is going to be created using ssh on crusoe.inria.fr, waha.inria.fr, nahuel.inria.fr.
Check examples/SSHList_example.xml or examples/SSHListbyHost_Example.xml for list examples.
LSF
This protocol is used to create Nodes(JVMs) on a cluster.
ProActive provides
org.objectweb.proactive.core.process.lsf.LSFBSubProcess
to create bsub process.
In this part we assume that you want to submit a job from a machine which is not the cluster frontend. As described before, you can combine protocols. In this case , you will have to define a process to log on the front-end of the cluster(rlogin if your machine is on the same LAN than the cluster front-end, else ssh (Remember that to use ssh you will have to run some commands as explained above).
<jvm name='Jvm2'> <creation> <processReference refid='sshProcess'/> </creation> </jvm> ................................................... <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='bsubInriaCluster'> <bsubProcess class='org.objectweb.proactive.core.process.lsf.LSFBSubProcess'> <processReference refid='jvmProcess'/> <bsubOption> <hostlist>cluster_machine1 cluster_machine2<hostlist/> <processor>6</processor> <scriptPath> <absolutePath value='/home/ProActive/scripts/cluster/startRuntime.sh'/> </scriptPath> </bsubOption> </bsubProcess> </processDefinition> <processDefinition id='sshProcess'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='sea.inria.fr'> <processReference refid='bsubInriaCluster'/> </sshProcess> </processDefinition> </processes>
In this example, the JVM called Jvm2 will be created using ssh to log on the cluster front end. Then a bsub command will be generated thanks to the process defined by bsubInriaCluster. This bsub command will create Nodes on several cluster machines, since bsubInriaCluster references the jvmProcess defined process. All tags defined under <bsubOption> are not mandatory, but they can be very usefull. The <hostlist> tag defines possible candidates in the job attribution, if not set the job will be allocated among all cluster's machines. The <processor> tag defines the number of processor requested, if not set, one processor is requested. The <resourceRequirement> tag defines the expected number of processors per machine. For instance <resourceRequirement value='span[ptile=2]'/> ensures that 2 processors per machines will be used, whereas value='span[ptile=1]' forces that LSF allocates only only one processor per machine. It represents the -R option of LSF. At last <scriptPath> defines the path on the cluster front end of the script startRuntime.sh which is necessary to run ProActive on a cluster. This script is located under Proactive/scripts/unix/cluster. If not set the default location is set as ~/Proactive/scripts/unix/cluster.
It is exactly the same with rlogin instead of ssh.
If you want to submit the job directly from the cluster entry point, define only the bsubProcess, like in the above example and skip the ssh definition.
<jvm name='Jvm2'> <creation> <processReference refid='bsubInriaCluster'/> </creation> </jvm> ................................................... <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='bsubInriaCluster'> <bsubProcess class='org.objectweb.proactive.core.process.lsf.LSFBSubProcess' interactive='true' queue='short'> <processReference refid='jvmProcess'/> <bsubOption> <hostlist>cluster_machine1 cluster_machine2<hostlist/> <processor>6</processor> <scriptPath> <absolutePath value='/home/ProActive/scripts/unix/cluster/startRuntime.sh'/> </scriptPath> </bsubOption> </bsubProcess> </processDefinition> </processes>
Note that in the example above two attributes: interactive and queue appear. They are optional, and have a default value: respectively false and normal. They represent option in the bsub command: interactive mode, and the name of the queue.
Check also examples/SSH_LSF_Example.xml .
PBS
This protocol is used to create jobs on cluster managed by
PBS, PBSPro or Torque. ProActive provides
org.objectweb.proactive.core.process.pbs.PBSBSubProcess
to create pbs processes. As
explained for LSF you can combine protocols in order for instance to
log on the cluster's frontal with ssh, then to create nodes using
PBS, or you can also use only PBS without ssh if you are already
logged on the frontend. Example below shows how to combine an ssh
process to log on the cluster, then a PBS process that references a
jvmProcess in order to create nodes
on processors requested by PBS.
<jvm name='Jvm2'> <creation> <processReference refid='sshProcess'/> </creation> </jvm> ................................................... <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='pbsCluster'> <pbsProcess class='org.objectweb.proactive.core.process.pbs.PBSSubProcess'> <processReference refid='jvmProcess'/> <pbsOption> <hostsNumber>4</hostsNumber> <processorPerNode>1</processorPerNode> <bookingDuration>00:15:00</bookingDuration> <outputFile>/home1/rquilici/out.log</outputFile> <scriptPath> <absolutePath value='/home/ProActive/scripts/unix/cluster/pbsStartRuntime.sh'/> </scriptPath> </pbsOption> </pbsProcess> </processDefinition> <processDefinition id='sshProcess'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='frontend'> <processReference refid='pbsCluster'/> </sshProcess> </processDefinition> </processes>
Note that not all options are listed here, and some options mentionned in the example are optionnal: hostsNumber represents the number of host requested using pbs(default is 1), processorPerNode represents the number of processor per hosts requested(1 or 2, default is 1), bookingDuration represents the duration of the job(default is 1 minute), outputFile represents the file where to put the ouput of the job(default is specified by pbs), scriptPath represents the location on the frontend_host of the script pbsStartRuntime.sh(default is /user.home/ProActive/scripts/unix/cluster/pbsStartRuntime.sh).
Check also examples/SSH_PBS_Example.xml.
Sun Grid Engine
This protocol is used to create jobs on cluster managed by Sun
Grid Engine. ProActive provides
org.objectweb.proactive.core.process.gridengine.GridEngineSubProcess
to create grid engine processes. As
explained above you can combine protocols in order for instance to
log on the cluster's frontal with ssh, then to create nodes using
SGE, or you can also use only SGE without ssh if you are already
logged on the frontend. Example below shows how to combine an ssh
process to log on the cluster, then a SGE process that references a
jvmProcess in order to create nodes
on processors requested by SGE.
<jvm name='Jvm2'> <creation> <processReference refid='sshProcess'/> </creation> </jvm> ................................................... <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='sgeCluster'> <gridengineProcess class='org.objectweb.proactive.core.process.gridengine.GridEngineSubProcess'> <processReference refid='jvmProcess'/> <gridEngineOption> <hostsNumber>4</hostsNumber> <bookingDuration>00:15:00</bookingDuration> <scriptPath> <absolutePath value='/home/ProActive/scripts/unix/cluster/gridEngineStartRuntime.sh'/> </scriptPath> <parallelEnvironment>mpi</parallelEnvironment> </gridEngineOption> </gridengineProcess> </processDefinition> <processDefinition id='sshProcess'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='frontend'> <processReference refid='sgeCluster'/> </sshProcess> </processDefinition> </processes>
As mentionned previously, many options exist, and correspond to the main options specified in an SGE system. ScriptPath represents the location on the frontend_host of the script gridEngineStartRuntime.sh (default is /user.home/ProActive/scripts/unix/cluster/gridEngineStartRuntime.sh).
Check also examples/SSH_SGE_Example.xml.
OAR:
OAR is a cluster protocol developed at INRIA Alpes and used on
Grid5000. ProActive
provides
org.objectweb.proactive.core.process.oar.OARSubProcess
to use such protocol.As explained above you can combine protocols in
order for instance to log on the cluster's frontal with ssh, then to
create nodes using OAR, or you can also use only OAR without ssh if
you are already logged on the frontend. Example below shows how to
combine an ssh process to log on the cluster, then an OAR process
that references a jvmProcess in
order to create nodes on processors requested by OAR.
<jvm name='Jvm2'> <creation> <processReference refid='sshProcess'/> </creation> </jvm> ................................................... <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='oarCluster'> <oarProcess class='org.objectweb.proactive.core.process.oar.OARSubProcess'> <processReference refid='jvmProcess'/> <oarOption> <resources>node=2,weight=2</resources> <scriptPath> <absolutePath value='/home/ProActive/scripts/unix/cluster/oarStartRuntime.sh'/> </scriptPath> </oarOption> </oarProcess> </processDefinition> <processDefinition id='sshProcess'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='frontend'> <processReference refid='oarCluster'/> </sshProcess> </processDefinition> </processes>
As mentionned previously, many options exist, and correspond to the main options specified in an OAR system. ScriptPath represents the location on the frontend_host of the script oarStartRuntime.sh (default is /user.home/ProActive/scripts/unix/cluster/oarStartRuntime.sh).
Check also examples/SSH_OAR_Example.xml and examples/SSH_OARGRID_Example.xml.
PRUN:
PRUN is a cluster protocol developed at Amsterdam to manage
their cluster.
ProActive provides
org.objectweb.proactive.core.process.prun.PrunSubProcess
to use such protocol.
Check also examples/SSH_PRUN_Example.xml.
GLOBUS
Like ssh, using globus requires some steps to be performed. In particular the java COG Kit (no need for the whole GT) must be installed on the machine that will originates the RSL request. See COG Kit Installation for how to install the client kit. Then you have to initialize your proxy by running COG_INSTALLATION/bin /grid-proxy-init, you will be asked for a passphrase, it is the one you provided when requesting a user certificate at globus.org. Once these steps are performed you can run ProActive application using GRAM protocol.
ProActive provides
org.objectweb.proactive.core.process.globus.GlobusProcess
to create globus process.
<jvm name='Jvm2'> <creation> <processReference refid='globusProcess'/> </creation> </jvm> ................................................... <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='globusProcess'> <globusProcess class='org.objectweb.proactive.core.process.globus.GlobusProcess' hostname='globus1.inria.fr'> <processReference refid='jvmProcess'/> <environment> <variable name='DISPLAY' value='machine_name0.0'/> </environment> <globusOption> <count>10</count> </globusOption> </globusProcess> </processDefinition> </processes>
In this example, Jvm2 will be created using GRAM. An RSL request will be generated with informations provided in the descriptor. For instance, the <environment> tag is not mandatory, but for the globus host to export the DISPLAY on your machine, you can define the value in the descriptor as well as other environment variable, except the classpath(or java path,...) which must be defined in the jvmProcess referenced by globusProcess as explained before. <globusOption> is neither manatory. Default value for <count> element is 1. It represents the number of processor requested.
Check also examples/Globus_Example.xml.
UNICORE:
ProActive provides
org.objectweb.proactive.core.process.unicore.UnicoreProcess
to use such protocol.
Check also examples/Unicore_Example.xml.
ARC (NorduGrid):
ProActive provides
org.objectweb.proactive.core.process.nordugrid.NGProcess
to use such protocol.
To use ARC you will need to download the ARC Client
Check also examples/NorduGrid_Example.xml.
GLITE
ProActive provides
org.objectweb.proactive.core.process.glite.GLiteProcess
to use such protocol.
Check also examples/SSH_GLite_Example.xml.
MPI
ProActive provides
org.objectweb.proactive.core.process.mpi.MPIDependentProcess
to use such protocol. You have to couple this process with the
DependentListProcessDecorator explained below.
Check also examples/SSH_MPI_Example.xml.
<processDefinition id='mpiProcess'> <mpiProcess class='org.objectweb.proactive.core.process.mpi.MPIDependentProcess' mpiFileName='my_mpi_program'> <commandPath value='/usr/bin/mpirun' /> <mpiOptions> <hostsNumber>16</hostsNumber> <localRelativePath> <relativePath origin="user.home" value='/ProActive/scripts/unix' /> </localRelativePath> <remoteAbsolutePath> <absolutePath value='/home/user' /> </remoteAbsolutePath> </mpiOptions> </mpiProcess> </processDefinition> <processDefinition id='dependentProcessSequence'> <dependentProcessSequence class='org.objectweb.proactive.core.process.DependentListProcessDecorator'> <processReference refid='pbsProcess' /> <processReference refid='mpiProcess' /> </dependentProcessSequence> </processDefinition> <processDefinition id='sshProcess'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='frontend' > <processReference refid='dependentProcessSequence' /> </sshProcess> </processDefinition>
<?xml version='1.0' encoding='UTF-8'?> <ProActiveDescriptor xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:noNamespaceSchemaLocation='DescriptorSchema.xsd'> <virtualNodesDefinition> <virtualNode name='PenguinNode' property='multiple'/> <virtualNodesDefinition/> <deployment> <mapping> <map virtualNode='PenguinNode'> <jvmSet> <vmName value='Jvm1'/> <vmName value='Jvm2'/> <vmName value='Jvm3'/> <vmName value='Jvm4'/> </jvmSet> </map> </mapping> <jvms> <jvm name='Jvm1'> <creation> <processReference refid='jvmProcess'/> </creation> </jvm> <jvm name='Jvm2'> <creation> <processReference refid='jvmProcess'/> </creation> </jvm> <jvm name='Jvm3'> <creation> <processReference refid='sshInriaCluster'/> </creation> </jvm> <jvm name='Jvm4'> <creation> <processReference refid='globusProcess'/> </creation> </jvm> </jvms> </deployment> <infrastructure> <processes> <processDefinition id='jvmProcess'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'/> </processDefinition> <processDefinition id='jvmProcess1'> <jvmProcess class='org.objectweb.proactive.core.process.JVMNodeProcess'> <classpath> <relativePath origin='userHome' value='/ProActive/classes'/> <relativePath origin='userHome' value='/ProActive/lib/bcel.jar'/> <relativePath origin='userHome' value='/ProActive/lib/asm.jar'/> <relativePath origin='userHome' value='/ProActive/lib/jini-core.jar'/> <relativePath origin='userHome' value='/ProActive/lib/jini-ext.jar'/> <relativePath origin='userHome' value='/ProActive/lib/reggie.jar'/> ............. </classpath> <javaPath> <absolutePath value='/usr/local/jdk1.4.0/bin/java'/> </javaPath> <policyFile> <absolutePath value='/home/ProActive/scripts/proactive.java.policy'/> </policyFile> <log4jpropertiesFile> <absolutePath value='/home/ProActive/scripts/proactive-log4j'/> </log4jpropertiesFile> <ProActiveUserPropertiesFile> <absolutePath value='/home/config.xml'/> </ProActiveUserPropertiesFile> </jvmProcess> </processDefinition> <processDefinition id='bsubInriaCluster'> <bsubProcess class='org.objectweb.proactive.core.process.lsf.LSFBSubProcess'> <processReference refid='jvmProcess1'/> <bsubOption> <hostlist>cluster_group1 cluster_group2<hostlist/> <processor>4</processor> <resourceRequirement value='span[ptile=2]'/> <scriptPath> <absolutePath value='/home/ProActive/scripts/unix/cluster/startRuntime.sh'/> </scriptPath> </bsubOption> </bsubProcess> </processDefinition> <processDefinition id='sshInriaCluster'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='sea.inria.fr'> <processReference refid='bsubInriaCluster'/> </sshProcess> </processDefinition> <processDefinition id='globusProcess'> <globusProcess class='org.objectweb.proactive.core.process.globus.GlobusProcess' hostname='cluster.inria.fr'> <processReference refid='jvmProcess1'/> <environment> <variable name='DISPLAY' value='machine_name0.0'/> </environment> <globusOption> <count>10</count> </globusOption> </globusProcess> </processDefinition> </processes> </infrastructure> </ProActiveDescriptor>
This xml deployment descriptor shows how to deploy the Penguin application on several places. Two Nodes will be created locally. We can see that with the definition of Jvm1 and Jvm2. These JVMs will be created locally since they reference directly the process defined by jvmProcess. Jvm3 will be created on the cluster using ssh to log on sea.inria.fr (cluster entry point)and then bsub to request processors and to create jvms on each. Here, two nodes will be created on machines that belong to cluster_group1 or cluster_group2 since processor tag is set to 2, and the hoslist tag gives cluster_group1 cluster_group2 as candidates. At Last Jvm4 will be created using globus It will access cluster.inria.fr and request 10 processors. We can notice that two local processes were defined, the reason is that the first one jvmProcess will use default value for the classpath, java path and policyfile path, whereas for the second one jvmProcess1 , we need to define these value, since the home directory is different between the local machine, and globus and the cluster(home dir is the same on globus machines and on the cluster, that is why both processes reference the same local process: jvmProcess1).
Even if quite a lot of things can be configured in the xml files, sometimes you will have to perform additional steps to get everything working properly, it is the case when using ssh, or globus as seen before. In this example, DISPLAY variable is defined for the globus process, that means that we want the penguin icon to appears on the local machine, be carefull to authorize your X server to display such icons by running the following command before launching the application: xhost +cluster.inria.fr. On the cluster side you need to create under ~/.ssh a file called environment where you define the DISPLAY variable. If you are not familiar with ssh, see openSSH
This process is used when a process is dependent on an another process. The first process of the list can be any process but the second one must be a DependentProcess thus has to implement the org.objectweb.proactive.core.process.DependentProcess interface.
Check also examples/SSH_MPI_Example.xml.
<processDefinition id='dependentProcessSequence'> <dependentProcessSequence class='org.objectweb.proactive.core.process.DependentListProcessDecorator'> <processReference refid='pbsProcess' /> <processReference refid='mpiProcess' /> </dependentProcessSequence> </processDefinition> <processDefinition id='sshProcess'> <sshProcess class='org.objectweb.proactive.core.process.ssh.SSHProcess' hostname='frontend' > <processReference refid='dependentProcessSequence' /> </sshProcess> </processDefinition>
As mentionned previously, instead of creating jvms, ProActive gives the possibility to acquire existing jvms. To do so, as shown in the example below, a service must be referenced in the acquisition tag. At this point two services are implemented: RMIRegistryLookup: this service performs a lookup in an RMIRegistry at the url specified in the service definition to find a ProActiveRuntime(a jvm) with the given name. P2PService service allows when using ProActive's P2P infrastructure to get as many jvms as desired.
<?xml version='1.0' encoding='UTF-8'?> <ProActiveDescriptor xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:noNamespaceSchemaLocation='DescriptorSchema.xsd'> <componentDefinition> <virtualNodesDefinition> <virtualNode name='VnTest' property='multiple'/> </virtualNodesDefinition> </componentDefinition> <deployment> <mapping> <map virtualNode='VnTest'> <jvmSet> <vmName value='Jvm1'/> <vmName value='Jvm2'/> </jvmSet> </map> </mapping> <jvms> <jvm name='Jvm1'> <acquisition> <serviceReference refid='lookupRMI'/> </acquisition> </jvm> <jvm name='Jvm2'> <acquisition> <serviceReference refid='lookupP2P'/> </acquisition> </jvm> </jvms> </deployment> <infrastructure> <services> <serviceDefinition id='lookupRMI'> <RMIRegistryLookup url='//localhost:2020/PA_JVM1'/> </serviceDefinition> <serviceDefinition id='lookupP2P'> <P2PService nodesAsked='2'acq='rmi' port='2410' NOA='10' TTU='60000' TTL='10'> <peerSet> <peer>rmi://localhost:3000</peer> </peerSet> </P2PService> </serviceDefinition> </services> </infrastructure> </ProActiveDescriptor>
The RMIRegistryLookup service needs only an url to perform the lookup. Many options exist for the P2PService service: nodesAsked represents the number of JVMs to acquire, the peer element represents an entry point in the P2P system, many peers can be specified. Elements acq and port represent the communication protocol and the listening port for the P2P Service, if a P2P Service is already running with this configuration the descriptor will use this one else a new one is started. Chapter 35, ProActive Peer-to-Peer Infrastructure provides more information.
The example above shows a VirtualNode VnTest, that is mapped with two jvms, Jvm1 and Jvm2. Jvm1 represents one jvm that will be acquired using an RMI Lookup, Jvm2 represents two jvms that will be found in the P2P infrastructure, so at the end 3 acquired jvms are expected.
Fault Tolerance can also be defined at the service level. See Chapter 25, Fault-Tolerance for more information.
ProActive gives the ability to kill all JVMs and Nodes deployed with an XML descriptor with the method: killall(boolean softly) in class ProActiveDescriptor (this class's code is in core/descriptor/data/ProActiveDescriptor.java)
ProActiveDescriptor pad = ProActive.getProactiveDescriptor(String xmlFileLocation); //----------- Returns a ProActiveDescriptor object from the xml file pad.activateMappings(); ... pad.killall(false); //----------- Kills every jvms deployed with the descriptor
If softly is set to false, all jvms created when activating the descriptor are killed abruptely. If true a jvm that originates the creation of a rmi registry waits until registry is empty before dying. To be more precise a thread is created to ask periodically the registry if objects are still registered.
There is the possiblity to use only the infrastructure part in order to create processes. A Schema called ProcessSchema located in the examples directory allows to validate XML files for processes. ProActive provides also the ability to use all processes defined above without using XML Deployment Descriptor. You can programmatically create such processes.
In order to get familiar on how to create processes programmatically, see the Process package
org.objectweb.proactive.core.process
© 2001-2007 INRIA Sophia Antipolis All Rights Reserved