Table of Contents
The aggregator framework module consists of an Aggregating ServiceGroup framework which supports plugins as detailed below, as well as a number of standard plugins.
The aggregating servicegroup framework is designed to facilitate the collecting of information from or about WS-Resources (via plugin aggregator sources) and the feeding of that information to plugin aggregator sinks.
The framework provides for over-the-wire management of the list of registered resources (through a WS-ServiceGroup interface) and a Java API for connecting sources and sinks together.
In general (although this is not a hard requirement), aggregator sinks will be tied into a specific service implementation, while aggregator sources are more independent. (For example, the trigger and index services act as sinks)
A number of standard aggregator sources are provided, which implement the aggregator source API. These provide for collecting information from/about a WS-Resource by:
- WS-ResourceProperties poll operations
- WS-Notification subscription
- Execution of arbitrary executables
See Aggregator Sources Reference for more information about standard aggregator sources for GT 4.0.
There are two main Java interfaces in the aggregator framework.
- AggregatorSink - which is implemented by sinks that can receive data from the aggregator framework.
- AggregatorSource - which is implemented by sources that can feed data into the aggregator framework.
In addition, the AggregatorContent class is used when configuring an aggregator service
programmatically, and to represent the data published in the aggregator's
Entry
resource property.
All aggregator classes and interfaces are documented in the
aggregator Java API documentation
The aggregator framework builds on the WS-ServiceGroup and WS-ResourceLifetime specifications. Those specifications should be consulted for details on the syntax of each operation.
Each aggregator framework is represented as a WS-ServiceGroup (specifically, an AggregatorServiceGroup).
Resources may be registered to an AggregatorServiceGroup using
the AggregatorServiceGroup Add operation. Each registration will be
represented as a ServiceGroupEntry resource. Resources may be
registered to an AggregatorServiceGroup using the service
group add
operation, which will cause an entry to be
added to the service group.
The entry will include configuration parameters for the aggregator source; when the registration is made, the following will happen:
- The appropriate aggregation source and sinks will be informed,
- the aggregator source will begin collecting data and inserting it into the corresponding service group entry,
- and the aggregator sink will begin processing the information in the service group entries.
The method of collection by source and processing by the sink is dependent on the particular instantiation of the aggregator framework (see per-source documentation for source information and per-service documentation for sink information - for example the Index Service and the Trigger Service.)
add:
This operation is used to register a specified resource with the aggregator framework. In addition to the requirements made by the WS-ServiceGroup specification, the Content element of each registration must be an AggregatorContent type, with the AggregatorConfig element containing configuration information specific to each source and sink (documented in the Aggregator Administrator's Guide).
Entry:
This resource property publishes details of each registered resource, including both an EPR to the resource, the aggregator framework configuration information, and data from the sink.RegistrationCount:
This resource property publishes registration load information (the total number of registrations since service startup and decaying averages)
The aggregator framework throws standard WS-ServiceGroup, WS-ResourceLifetime, and WS-ResourceProperties faults and does not define any new faults of its own.
Other relevant source files are the:
- WSRF service group schema
- WSRF resource lifetime schema
- MDS Usefulrp schema.
Please see the Aggregator Command Reference.
There is no GUI specifically for the aggregator framework. The release contains WebMDS which can be used to display monitoring information in a web browser. Specifically, it can be directed at services based on the aggregator framework to display information about resources registered to the aggregator framework.
The execution aggregation source provides a way to aggregate data (arbitrary XML information) about a registered resource using an arbitrary local executable (such as an external script). The executable will be passed registration information as parameters and is expected to output the gathered data, as detailed below.
A basic example of the use of this API is described in the ping test example for the aggregator execution source
The execution aggregation source will periodically execute an identified executable. The identity of the executable and the frequency with which it is to run are specified in the registration message.
To register resources:
- Create a configuration file in XML that specifies registrations. See
$GLOBUS_LOCATION/etc/globus_wsrf_mds_aggregator/example-aggregator-registration.xml
for several specific examples. - Run mds-servicegroup-add to perform the registrations specified in that configuration file.
The file consists of an optional defaultServiceGroupEPR
, an
optional defaultRegistrantEPR
, and then one or more
ServiceGroupRegistrationParameters
blocks, each of which
represents one registration.
The general syntax of the configuration file is:
<?xml version="1.0" encoding="UTF-8" ?> <ServiceGroupRegistrations xmlns="http://mds.globus.org/servicegroup/client"> // An optional default service group EPR. <defaultServiceGroupEPR> // Default service group EPR </defaultServiceGroupEPR> // An optional default registrant EPR. <defaultRegistrantEPR> // Default registrant EPR </defaultRegistrantEPR> // An optional default security descriptor file. <defaultSecurityDescriptorFile> // Path name of default security descriptor file </defaultSecurityDescriptorFile> // One or more service group registration blocks: <ServiceGroupRegistrationParameters> <ServiceGroupEPR> // EPR of the service group to register to </ServiceGroupEPR> <RegistrantEPR> // EPR of the entity to be monitored. </RegistrantEPR> <InitialTerminationTime> // Initial termination time </InitialTerminationTime> <RefreshIntervalSecs> // Refresh interval, in seconds </RefreshIntervalSecs> <Content type="agg:AggregatorContent"> // Aggregator-source-specific configuration parameters </Content> </ServiceGroupRegistrationParameters> </ServiceGroupRegistrations>
The following table describes the different blocks of the file and any parameters:
Table 1. Aggregator configuration parameters
defaultServiceGroupEPR block | The provides a convenient way to register a number
of resources to a single service group -- for example, if you wish
to register several resources to your default VO index, you can
specify that index as the default service group and omit the
ServiceGroupEPR blocks from each
ServiceGroupRegistrationParameters block. |
defaultRegistrantEPR | The provides a convenient way to register a single
resource to several service groups -- for example, if you wish to
register your local GRAM server to several index servers, you can
specify your GRAM server as the default registrant and omit the
RegistrantEPR blocks from each
ServiceGroupRegistrationParameters block. |
defaultSecurityDescriptorFile | Simply the path to the security descriptor file. |
ServiceGroupRegistrationParameters | Each
ServiceGroupRegistrationParameters block specifies the
parameters used to register a resource to a service group. The
parameters specified in this block are: |
ServiceGroupEPR | The EPR of the service group to register to. This parameter may
be omitted if a defaultServiceGroupEPR block is
specified; in this case, the value of
defaultServiceGroupEPR will be used instead. |
RegistrantEPR | The EPR of the resource to register. This parameter may be
omitted if a defaultRegistrantEPR block is specified;
in this case, the value of defaultRegistrantEPR will
be used instead. |
InitialTerminationTime | The initial termination time of this registration (this may be
omitted). If the initial termination time is omitted, then the
mds-servicegroup-add sets the initial termination time to the
current wall time plus 2 times that of the specified
RefreshIntervalSecs parameter. |
RefreshIntervalSecs | The refresh interval of the registration, in seconds. The mds-servicegroup-add will attempt to refresh the registration according to this interval, by default incrementing the termination time of the registration by 2 times this interval for every successful refresh. If at any point the termination time for the registration expires the registration will be subject to removal within a maximum of 5 minutes. |
Content | Aggregator-source-specific registration parameters. The content blocks for the various aggregator sources are described in detail in the following sections. |
The configuration block for ExecutionAggregatorSource
(inside the Content
block) looks like this:
<Content xsi:type="agg:AggregatorContent" xmlns:agg="http://mds.globus.org/aggregator/types"> <agg:AggregatorConfig xsi:type="agg:AggregatorConfig"> <agg:ExecutionPollType> <agg:PollIntervalMillis>interval_in_ms
</agg:PollIntervalMillis> <agg:ProbeName>dummy_namespace
:probe_name
</agg:ProbeName> </agg:ExecutionPollType> </agg:AggregatorConfig> <agg:AggregatorData/> </Content>
where:
PollIntervalMillis
This parameter is the poll refresh period in milliseconds.
ProbeName
This parameter specifies name of the probe to run. This probe is defined in the
jndi-config.xml
file for the service being configured (for example, the file for the MDS Index service is$GLOBUS_LOCATION/etc/globus_wsrf_mds_index_jndi-config.xml
). AnexecutableMappings
parameter should be defined within this file to map probe names to executable names. For example, this maps the probe namesaggr-test
andpingexec
to the executables calledaggregator-exec-test.sh
andexample-ping-exec
, respectively. All executables are presumed to be in the directory$GLOBUS_LOCATION/libexec/aggrexec
.<resource name="configuration" type="org.globus.mds.aggregator.impl.AggregatorConfiguration"> <resourceParams> // ... <parameter> <name>executableMappings</name> <value>aggr-test=aggregator-exec-test.sh, pingexec=example-ping-exec</value> </parameter> </resourceParams> </resource>
If you've properly configured and registered your script for execution but are getting errors from the container because it cannot find the specified script, there are two likely causes.
First, make sure that your script/program is executable and is located in the
$GLOBUS_LOCATION/libexec/aggrexec
directory.
When it's specified in the configuration mentioned above, only specify
the name of the script/program, without any qualification or path.
For example, using the ProbeName
as
test-script
will be specifying the file
$GLOBUS_LOCATION/libexec/aggrexec/test-script
script.
Next, make sure that you have correctly created an
executableMappings
definition in the appropriate
jndi-config.xml
file.
The executable to run will be
$GLOBUS_LOCATION/libexec/aggrexec/<scriptname>
with scriptname
supplied by the ProbeName
parameter in the configuration file.
Information about the registration will be supplied as command line parameters and on stdin.
A single command line parameter will be supplied to the executable. This will be the URL from the EPR of the registered service.
Two XML documents will be sent to stdin, in sequence:
- The first document will be the full EPR to the registered service.
- The second document will be the AggregatorConfig block from the registration message (configuration file).
The aggregator framework does not have its own service side configuration, although services which are based on the framework have their own service side configuration options (such as MDS Index and MDS Trigger) which are documented in the per-service documentation.
Registrations to a working aggregator framework are configured for the mds-servicegroup-add tool. This tool takes an XML configuration file listing registrations, and causes those registrations to be made.
In general, configuration of aggregator services involves configuring the service to get information from one or more sources in a Grid. The mechanism for doing this is defined by (inherited from) the aggregator framework and described in this section.
Configuring an Aggregating Service Group to perform a data aggregation is performed by specifying an AggregatorContent object as the content parameter of a ServiceGroup add method invocation. An AggregatorContent object is composed of two xsd:any arrays: AggregatorConfig and AggregatorData:
- The AggregatorConfig xsd:any array is used to specify parameters that are to be passed to the underlying AggregatorSource when the ServiceGroup add method is invoked. These parameters are generally type-specific to the implementation of the AggregatorSource and/or AggregatorSink being used.
- The AggregatorData xsd:any array is used as the storage location for aggregated data that is the result of message deliveries to the AggregatorSink. Generally, the AggregatorData parameter of the AggregatorContent is not populated when the ServiceGroup add method is invoked, but rather is populated by message delivery from the AggregatorSource.
The following links provide information for configuring the three types of aggregator sources provided by the Globus Toolkit:
An aggregator sink may require sink-specific configuration (the MDS Trigger service requires sink-specific configuration; the MDS Index service does not). See the documentation for the specific aggregator service being used for details on sink-specific documentation.