| Prev | Part V. Deployment And Virtualization Chapter 18. ProActive Grid Component Model Deployment |
 |
Next |
| Prev | Part V. Deployment And Virtualization Chapter 18. ProActive Grid Component Model Deployment |
|
Next |
The GCM Deployment is split in two parts, one for grid administrators and the other for grid application developers. On the grid administration side, the administrator will write a Deployment Descriptor that will describe what resources the grid provides, and how these resources are acquired. On the application side, the developer will write an Application Descriptor that will describe how the application is launched, and what resources it needs. The link between the two sides is made through references from the Application Descriptor to one or several Deployment Descriptors.
There are several ways the grid resources can be used by a deployed application. The application may require a fixed set of resources, or it may be flexible enough to work on any amount of resources, or finally may require a minimum amount of resources and yet be able to scale as more resources become available.
In all cases, the application must start by creating a GCMApplication object through
PAGCMDeployment.loadApplicationDescriptor(), and call GCMApplication.startDeployment(). The
application must quit through GCMApplication.kill().
In this case the application knows the amount of resources it requires. The acquisition of these resources by the application is done as follows :
get the required virtual nodes through GCMApplication.getVirtualNode(String vnName),
or GCMApplication.getVirtualNodes()
For each virtual node, use GCMVirtualNode.getNewNodes() as many times as needed,
until the virtual node has the expected numbers of physical nodes to run on.
getNewNodes() will return the list of Nodes that have been acquired since the last time
it was called. Calls to it should be separated by calls to Thread.sleep().
In this case the application has no specific requirement on the resources it uses : the more the better.
This is the simplest of the cases : the application only has to call GCMApplication.waitReady().
This will block until all Virtual Nodes have their configured number of physical Nodes.
Note that this may block forever if a Virtual Node does not to have a limited number of nodes after which it
is in 'ready' state (the Virtual Node is said to be 'greedy', GCMVirtualNode.isGreedy() will return true).
In the case the application is able to expand on new resources as they become available.
This is a extension of the two other cases, in that it can work whether the application has fixed minimum requirements
or not. Once the initial deployment phase is finished, the application should call GCMApplication.getVirtualNodes()
to obtain the list of configured virtual nodes,
and then subscribe to the node attachment notifications for each of them (GCMVirtualNode.subscribeNodeAttachment() ).
In the notification handler, the application should deal with the newly acquired node appropriately.
The HostInfo data structure describes a single machine and the environment it provides, with the following information:
userName : (string) the name of the user under which this host can be accessed
homeDirectory : (absolute path) the home directory of the user
os : (one of "unix" or "windows") the operating system the host is running
hostCapacity : (positive integer) the number of processes (VM or other executable) that this host can handle (default value is 1)
vmCapacity : (positive integer) the number of nodes a single VM on this host can handle (default value is 1)
id : (ID) an ID identifying the host
A bridge is meant to represent a frontend to a computing resource. Many grid architectures have such a feature : each physical machine is not accessible directly, the user must instead go through a single machine called a front-end. In a deployment descriptor, a bridge is a gateway toward either :
a host
a set of groups
another bridge
A bridge is defined as a base structure meant to be derived. The base structure only defines an id (string).
An RSH bridge element can have the following attributes :
id (string) : the id of the bridge connector corresponding to this definition
hostname (string) : the network hostname of the physical machine which acts as the bridge
username (string, optional) : the user name under which the machine can be accessed
commandPath (string, optional) : the path of rsh client to use
An SSH bridge element can have the follow child element :
privateKey (path string) : the file of the private SSH key needed to access the bridge
An SSH bridge element can have the following attributes :
id (string) : the id of the bridge connector corresponding to this definition
hostname (string) : the network hostname of the physical machine which acts as the bridge
username (string, optional) : the user name under which the machine can be accessed
commandPath (string) : the path of the ssh client to use
commandOptions (string) : options to pass to the ssh command
A Group is a data structure defining a set of machines with identical configuration (like a cluster). It is meant as a base structure which can be derived in an Object-Oriented manner to implement any kind of group. There currently are two kinds of groups :
"direct" groups
job schedulers
It is therefore possible to define a standard-compliant deployment descriptor even on a grid which has its own job scheduler.
All group protocols have the following child elements :
environment (environment) : the environment for the command and the following attributes
and the following attributes
id (ID) : the id of the group this element represents
commandPath (path string) : path of the command which is used to submit a job to the group protocol
This group handles Microsoft’s Compute Cluster Server. The CCS group definition has the following child elements :
resources : the resources that will be allowed to the job. This element can have two children :
cpus (positive integer) : the number of CPUs allocated for the job
runtime (time) : the maximum runtime allowed for the job
stdout (path string) : path of the file where the standard output of the job will be stored
stderr (path string) : path of the file where the standard error of the job will be stored
Group definition for the LSF scheduler. The LSF group definition has the following child elements :
resource (string) : this element has the following attributes :
(positive integer) : number of processors requested
walltime (time) : maximum time allowed for the job
processorsNumber (positive integer) : minimum number of processors requested to run the job
resourceRequirement (string) : a resource requirement string as defined by the lsf documentation (‘lsfintro’ manpage)
It also has the following attributes
interactive (boolean) : whether the job is interactive or not
jobName (string) : name of the job
queue (string) : name of the queue the job will be submitted in
Group definition for the OAR job scheduler [15]. The OAR group definition has the following child elements :
resource (string) : this element has the following attributes :
nodes (positive integer) : number of nodes requested
cpu (positive integer) : number of CPUs requested
core (positive integer) : number of cores requested
walltime (time) : maximum time allowed for the job
It also can have a string content which is passed verbatim–o the ‘--resource’ option of the oarsub command.
directory (path string) : the working directory of the job script
stdout (path string) : path of the file where the standard output of the job will be stored
stderr (path string) : path of the file where the standard error of the job will be stored
It has the following attributes :
interactive (boolean) : start an interactive job. If true, open a login shell on the first node instead of running a script (default is false).
queue (string) : name of the queue to submit the job to.
type (‘deploy’, ‘besteffort’, ‘cosystem’, ‘checkpoint’, ‘timesharing’) : job type – the default is ‘deploy’.
Group definition for the PBS/Torque job scheduler. The PBS/Torque group definition has the following child elements :
resource (string) : this element has the following attributes :
nodes (positive integer) : number of nodes requested
ppn (positive integer) : number of CPUs requested
walltime (time) : maximum time allowed for the job
nodes (positive integer) : number of nodes requested
processorsPerNode (positive integer) : number of processors per node requested
mailWhen (combination of Abort, Begin, End separated by ‘|’) : when to send an email (Abort : if the job is aborted, Begin : when the job is started, End : when the job terminates)
mailTo (comma-seperated list of email addresses) : where the job status emails should be sent
joinOutput (boolean) : if true, join the output of stderr to stdout
stdout (path string) : path of the file where the standard output of the job will be stored
stderr (path string) : path of the file where the standard error of the job will be stored
It has the following attributes :
queue (string) : destination queue for the job. The argument can be of the following format :
queue : a queue on the default server
@server : the default queue on the server
queue@server : the queue on the given server
jobName (string 15 chars long, no whitespace, first char must be alphabetic) : the name of the job
interactive (boolean) : whether the job is interactive or not
Group definition for the PRUN run server. The PRUN group definition has the following child elements :
resource (string) : this element has the following attributes :
nodes (positive integer) : number of nodes requested
ppn (positive integer) : number of CPUs requested
walltime (time) : maximum time allowed for the job
stdout (path string) : name of the file in which the results will be printed
It has no attribute.
A host list can be used with SSH and RSH groups as a shorthand to specify several machine names in a compact form. The format of a host list is a whitespace-separated list of name patterns or hostnames. A name pattern describes a set of hostnames with a common root. The format is as follows.
<root name><interval>
with root name being an alphanumeric string (only letters and digits, no spaces or punctuation signs), and interval defining a set of numerical values in the form of an interval or list of values, possibly followed by an exclusion interval or list of values. The general form of an interval is:
[<value set>]^[<value set>]
or simply
[<value set>]
if no exclusion interval is needed.
A value set is a coma-seperated list of integers or integers pairs separated by a dash, meaning an interval of values. The values of an interval must be specified in increasing order, and the generated values will be in increasing order. Also, the first integer of an interval can have leading zeroes to indicate the number of digits (numbers will be padded with zeroes if needed). Some examples:
host[0-5]: host0, host1 … host5;
host[0-5]^[4]: host0, host1, host2, host3, host5;
host[0-10]^[4-6]: host0, host1, host2, host3, host7, host8, host9, host10;
host[00-5]: host00, host01, host02… host05;
host[1, 004-7, 09]: host1, host004, host005, host006, host007, host09.
The RSH Group has the following child elements :
(host list) : the list of hosts to connect to
This type of application describes the launch of a stand-alone executable on the grid. It can have the following child elements :
nodeProvider (empty element with a single ‘refId’ attribute) : the id of a node provider (defined in the <resources> part). There can be any number of such element.
command : the command which will be run on the portion of the grid defined by the specified node providers. The contents of this element are described below.
This element can have the following attribute :
instances (one of “onePerHost”, “onePerVM”, “onePerCapacity”) : the number of instances of the command which will be run
The <command> element can have the following children (in this specified order) :
path (path string) : the path of the executable
arg (string) : the arg string which will be passed to the command. There can be any number of such element.
filetransfer (file transfer) : the files which which should be transferred prior to running the command.
It can have the following attribute :
name (string) : name of the executable. If a <path> child element is present, the value of this attribute will be appended to the value of the %lt;path> child element.
This element describes a ProActive-based applicatin. It can have the following children :
configuration : various configuration parameters - this element is described below
technicalServices (technical services) : the set of technical services global to this instance of ProActive
the configuration element can have the following child elements :
bootClasspath (simple classpath) : the boot classpath for the JVM
java (path string) : the path to the Java executable
jvmarg (string) : arguments passed to the JVM
applicationClasspath (classpath) : classpath for the application
proactiveClasspath (classpath) : classpath used to override the standard ProActive classpath computed from its installation location
securityPolicy (relative path) : path to the Java security policy file
proactiveSecurity : security policy for application and runtime. This element has two children :
applicationPolicy (relative path) : path to Java security policy file that will be applied on the application's objects deployed at runtime, like nodes and active objects
runtimePolicy (relative path) : path to Java security policy file that will be applied on the ProActive Runtime
log4jProperties (relative path) : path to the Java log4j configuration file
userProperties (relative path) : path to the Java properties file
virtualNode (virtual node) : description of a virtual node. There can be any number of such element
The <proactive> element can have the following attributes :
relpath (path string) : the location of the ProActive installation
base (one of ‘HOME’, ‘ROOT’) : base location of the ProActive installation : HOME is the user’s home directory, ROOT is the root directory of the system.
A <virtualNode> element can have the following children :
nodeProvider (reference to a node provider) : the node provider which will provide the ProActive nodes for this virtual node – see below for description.
technicalServices (technical service) : a technical service specific to this virtual node. There can be any number of such children.
A virtualNode element can also have the following attributes :
id (string) : a string identifying this virtual node
capacity (positive integer) : the capacity requested by this virtual node (that is, the total number of nodes it will request from the node providers which are affected to it). If no capacity is specified, then the Virtual Node will try to get as many nodes as possible. A such Virtual Node is called greedy.
A <nodeProvider> within a <virtualNode> can only have <technicalServices> child elements. These describe technical services specific to this node provider. A <nodeProvider> can also have the following attributes :
refid (string) : the id of the node provider (as defined in the resources element)
capacity (positive integer or “max”) : the capacity of this ProActive node provider (that is, the number of ProActive nodes which will be requested from it)
This tutorial shows how to deploy a grid-enabled application through the GCM standard. It will present the points of view of both the grid administrator and the application developer.
The task of a grid administrator is to make a model of his grid resources through a GCM Deployment Descriptor. Several examples are available in the ProActive distribution. The deployment descriptor should represent the resources of the grid. A deployment descriptor has the following XML structure:
<environment> <descriptorVariable …/> … </environment> <resources> <bridge …/> <group> <host …/> <host …/> … </group> … </resources> <acquisition> <lookup …/> <p2p …/> … </acquisition> <infrastructure> <hosts> <host…/> </hosts> <groups> <groupType …/> </group> <bridges> <bridgeType …/> </bridges> … </infrastructure>
The elements must be specified in this order. The <environment> and <acquisition> elements can be omitted, while the <resources> and <infrastructure> ones are mandatory. They are the ones which define the model :
Infrastructure: this is a flat list of each individual element of the grid: hosts, groups and bridges listed in no particular order.
Resources: this is a tree describing the hierarchical relationships between these infrastructure elements. These relationships are defined by:
a host being within which group;
a group being behind a bridge;
a host being directly available.
Let's examine a couple of basic examples. Considering a very simple grid, that is two desktop PCs networked together. Such a setup would be represented as follows (configuration parameters are omitted for the sake of clarity):
<resources> <hosts> <host refid="host1" /> <host refid="host2" /> </hosts> </resources> <infrastructure> <hosts> <host id="host1" /> <host id="host2" /> </hosts> </infrastructure>
There is no hierarchical relation between the two hosts, so both resources and infrastructure parts are identical (aside of the extra configuration parameters which are omitted here). A slightly more complex example would be a cluster of 12 mono-processor machines running LSF. The representation in GCM Deployment would be as follows:
<resources> <group refid="LSF_GROUP"> <host refid="LSF_GROUP_MEMBER" /> </group> </resources> <infrastructure> <hosts> <host id="LSF_GROUP_MEMBER" /> </hosts> <groups> <lsfGroup id="LSF_GROUP" > <resources processorNumber="12" /> </lsfGroup> </groups> </infrastructure>
Within the <infrastructure>, the <hosts> part describes the configuration common to the machines in the group. The <groups> part describes the LSF group itself. Finally, the <resources> part describes how they fit together, in this case the host model being within the LSF group.
The next paragraphs go more in depth on the content and usage of each element.
To allow for a bit of flexibility, it is possible to define variables in a descriptor. The variables can be used in any XML value element. They cannot be used in an XML element name. The <environment> element is where the variables are defined. It is a simple list of <descriptorVariable> elements. For example:
<environment> <descriptorVariable name="usertype" value="admin" /> <descriptorVariable name="username" value="jsmith" /> </environment>
This allows the following usage later on in the descriptor: <sshGroup user="${username}" />
The <resource> element describes the hierarchical structure of the available grid resources. This can be seen as the topology of the grid: which hosts are part of a group, which group is behind a bridge, etc… All the grid resources which are listed in it must be fully defined in the <infrastructure> element. However it doesn't have to hold every element listed in <infrastructure>, it is meant to contain only the subset of resources which are actually used by the deployment.
You can use the following elements to build your grid topology :
<host> : this represents a single machine, or more precisely a single configuration. When used within a group, it represents the common configuration of all machines within this group.
<group> : this represents a set of machines all sharing a common configuration. Typically a cluster. The configuration is represented through a Host element.
<bridge> : this represents a machine which acts as a gateway to one or several other machines. Typically, a front-end for a cluster.
These elements all take a single argument named 'refid'. The value of the argument is the id of the corresponding host/bridge/group element defined in the <infrastructure> element. The topology must be described according to the following rules:
A host can be at the top level, or in a group element
A group can be at the top level, or in a bridge element
A bridge can only be at the top level
For example, the following constructions are correct: Single host:
<host refid="A_HOST" />
Group:
<group refid="CLUSTER"> <host refid="CLUSTER_NODE" /> </group>
Group behind a bridge:
<bridge refid="CLUSTER_FRONT_END" /> <group refid="CLUSTER"> <host refid="CLUSTER_NODE" /> </group> </bridge>
An alternative to the <infrastructure> element, the <acquisition> element describes how resources which are already running can be acquired. It contains two types of elements: <lookup> and <p2p>, in this order. Each element can either have a single occurrence or be omitted. The <lookup> element has the following three attributes:
type: one of "RMI", "HTTP", "IBIS";
hostlist: a HostList as defined in 5.1.1;
port: a positive integer.
The <p2p> element has a single attribute named "nodesAsked", indicating the number of requested nodes. You can set the value to 'MAX' so that the maximum number of available nodes will be allocated to the task.
The <infrastructure> is where you will list the grid resources on which the deployment can take place, in no particular order. Its purpose is to describe how these resources are deployed (i.e. through which protocols). It can have a single child element of each of the following types: <hosts>, <bridges>, <groups>. <bridges> and <groups> may be empty or omitted, but there should be at least one child element in <hosts>.
While the Deployment Descriptor lists the grid resources, the application descriptor lists the resources the application needs.
The overall structure of an Application Descriptor is as follows :
<environment> … </environment> <application> … </application> <resources> <nodeProvider> <file …/> … </nodeProvider> … </resources>
The <environment> element is similar to the one in the Deployment Descriptor. The <application> one is where the application itself and the resources it requests are described (see GCM Application descriptor section). Finally, the <resources> element is where you'll make the link between the requested resources and the deployed ones.
The <application> tag can hold either an <executable> or a <proactive> tag. <executable> is for stand-alone applications which you want to run on a grid. <proactive> is for ProActive-based applications. In both cases the requested resources are specified through <nodeProvider> elements. These elements only carry a single 'refid' attribute which points to a corresponding <nodeProvider> element listed in the <resources> element.
A stand-alone executable is very straightforward to describe. You only need to specify one or several <nodeProvider>s and the application will be run on all the physical nodes these providers can yield.
<application> <executable> <command name="ls"> <arg>-lh</arg> <arg>--sort=time</arg> <arg>*</arg> </command> <nodeProvider refid="COMPANY_LAN" /> </executable> </application> <resources> <nodeProvider id="COMPANY_LAN"> <file path="deployment.xml" /> </nodeProvider> </resources>
A ProActive application runs on virtual nodes. These virtual nodes aggregate physical nodes that they fetch from the node provider specified in the virtual node definition. In the following example, a ProActive application defines two virtual nodes ("master" and "slaves"). The first will fetch a single physical node from the COMPANY_LAN node provider. The second will get as many nodes as available (its capacity is set to "MAX") from both the COMPANY_LAN and REMOTE_CLUSTER providers.
<application> <proactive relpath="Scratch/ProActive/"> <configuration> <!-- ommitted for clarity --> </configuration> <virtualNode id="master" capacity="1"> <nodeProvider refid="COMPANY_LAN" /> </virtualNode> <virtualNode id="slaves" capacity="max"> <nodeProvider refid="COMPANY_LAN" /> <nodeProvider refid="REMOTE_CLUSTER" /> </virtualNode> </proactive> </application> <resources> <nodeProvider id="COMPANY_LAN"> <file path="deployment.xml" /> </nodeProvider> <nodeProvider id="REMOTE_CLUSTER"> <file path="deployment_cluster.xml" /> </nodeProvider> </resources>
To deploy your application on the grid, you need to get your application descriptor as a
java.io.File
. You then pass it to
org.objectweb.proactive.extensions.gcmdeployment.PAGCMDeployment.loadApplicationDescriptor()
which will return a GCMApplication object. To actually start the deployment, simply call the
startDeployment()
method.
GCMApplication app; File desc = new File(this.getClass().getResource("MyApplicationDescriptor.xml").getPath()); app = PAGCMDeployment.loadApplicationDescriptor(desc); app.startDeployment();
If needed you may want to also create a VariableContract and set some of its variables, then pass it as 2nd argument to loadApplicationDescriptor() :
VariableContractImpl vContract = new VariableContractImpl(); vContract.setVariableFromProgram("HOST_CAPACITY", "4", VariableContractType.DescriptorDefaultVariable); vContract.setVariableFromProgram("VM_CAPACITY", "1", VariableContractType.DescriptorDefaultVariable); GCMApplication app; File desc = new File(this.getClass().getResource("MyApplicationDescriptor.xml").getPath()); app = PAGCMDeployment.loadApplicationDescriptor(desc, vContract); app.startDeployment();
In the case of a stand-alone application, it will simply be deployed on all the available nodes without any special intervention on your side. In the case of a ProActive-based application, there are two ways for an application to handle the deployment process. The simplest one is to call the
public void waitReady();
method on your GCMApplication object. As the name of the method indicates, it amounts to "wait until everything is ready". The call
will block until all virtual nodes are ready, that is that they have acquired the minimum number of nodes they need. This
method
should not be used if one of the virtual nodes is "greedy", in which case it will never be in a "ready" state. There's
another version
of the method with a timeout parameter :
public void waitReady(int timeout) throws ProActiveTimeoutException;
However, a more flexible way is for your application to listen to the availability of new nodes on each virtual node, and act accordingly. The method for this is
public void subscribeNodeAttachment(Object client, String methodName, boolean withHistory)
in
GCMVirtualNode
.
methodName
must be the name of a method of
client
, which prototype is
method(GCMVirtualNode node, String virtualNodeName)
. This method will be called by the virtual node for each new available node.
To get the list of virtual nodes for your GCMApplication, use
public Map<String, GCMVirtualNode> getVirtualNodes();
You can also get a specific virtual node if you know its name :
public GCMVirtualNode getVirtualNode(String vnName);
© 1997-2008 INRIA Sophia Antipolis All Rights Reserved