Barrier
Jobflow Component
| Short Description |
| Ports |
| Metadata |
| Barrier Attributes |
| Details |
| Examples |
| See also |
Short Description
Barrier allows to wait for results of parallel running jobs and react to the success or failure of groups of jobs in a simple way.
Barrier waits for all input tokens belonging to a group, evaluates this group and based on the results sends output token(s) to the first or second output port.
![[Note]](figures/note.png) | Note |
|---|---|
To be able to use this component, you need a separate jobflow license. Also, the component requires your project is executed on CloverETL Server. |
| Component | Same input metadata | Sorted inputs | Inputs | Outputs | Each to all outputs | Java | CTL | Auto-propagated metadata |
|---|---|---|---|---|---|---|---|---|
| Barrier |  | | 1-n | 1-2 | | | | |
Icon
Ports
| Port type | Number | Required | Description | Metadata |
|---|---|---|---|---|
| Input | 0 |  | Input tokens with job results. | Any |
| 1-n | | Input tokens with job results. | Any | |
| Output | 0 | | Tokens for successful groups of jobs. | Any |
| 1 | | Tokens for unsuccessful groups of jobs. | Any |
Metadata
Input ports
Any metadata are possible on the input ports, only field called 'status' is expected by 'Token evaluation expression' attribute by default ($status == "FINISHED_OK").
Output ports
Any metadata are possible on the output ports, but tokens sent to output ports are copied based on field names from input ports, so only fields with equal names are populated.
Barrier Attributes
| Attribute | Req | Description | Possible values |
|---|---|---|---|
| Basic | |||
| Input grouping | no | Type of algorithm how the incoming tokens are split into groups of jobs, which are evaluated independently. See Logical grouping of incoming records | Tuple (default) | All |
| Token evaluation expression | no | Boolean CTL expression which is applied to each incoming token to decide, whether the token represents successful or unsuccessful job run - final job status. See Group evaluation | default CTL expression '$status == "FINISHED_OK"' |
| Group evaluation criteria | no | Logical operation which is applied to the job status (see attribute 'Token evaluation expression') to decide, whether the group of jobs is successful or unsuccessful. See Group evaluation | AND (default) | OR |
| Output | no | Defines number of output tokens per group of jobs. See Generating output tokens | Single token (default) | All tokens |
Details
Barrier is mainly used for management of parallel running jobs. Barrier receives all incoming tokens, which carry information about job results, and splits them to logical groups of jobs. Each job group is evaluated independently. Results of groups evaluated as successful are sent to the first output port. Results of the unsuccessful groups are sent to the second output port.
Logical grouping of incoming records
Component attribute 'Input grouping' provides two options how the incoming tokens are split to logical groups of jobs.
All - all incoming tokens are considered as a single group, exactly one group is processed by the component. This covers the most common scenarios, e.g. checking that all previous jobs were successful.
Tuple - a group consists of a single token from all input ports, i.e. the groups are created by "waves" of tokens coming from input ports. Tokens which arrive first from each input port form the first logical group, the second tokens from each input port form the second logical group etc (i.e. n-th group consists of n-th input token from all input ports). This setting covers checking result of waves of parallel job.
Group evaluation
Each token in a group is evaluated by CTL boolean expression from component attribute 'Token evaluation expression' - let's call it job status. The group is considered successful if and only if the job statuses joined by logical operation AND or OR (component attribute 'Group evaluation criteria') are true. So in case of AND operation, all incoming token needs to be successful for success of whole group. On the other hand in case of OR operation, at least one token from the group needs to be successful for group success.
Generating output tokens
Successful groups send their results to the first output port and the unsuccessful groups send their results to the second output port. Number and content of output tokens is specified by component attribute 'Output':
Single token - only one token is sent to an output port per group, the token is populated by all group tokens - fields are copied based on fields names (input tokens order to be copied is not guaranteed).
All tokens - each incoming token from the group is sent to the dedicated output port, in case of incompatible metadata fields are copied based on field names.
| Note |
|---|---|
Output ports are not required. Tokens routed to a missing edge are quietly discarded. |
Examples
Let's look at simple example of usage.
Figure 52.1. Example of typical usage of Barrier component
In this example, three different graphs are synchronously executed by three ExecuteGraph components. All three graphs are running in parallel. Barrier is a collection point for graph execution outcomes; it waits for all graphs to finish prior to moving on to the next step. If all graphs finished successfully, an output token is sent to the first output port. On the other hand, if one or more graphs failed, an output token is sent to the second output port. This component allows simple evaluation for status of the whole job group.