Home | Table of Contents | ExecuteGraph | CloverETL 4.6.1
--- | --- | ---
Prev | Job Control | Next

## ExecuteGraph

# Jobflow Component

Short Description
Ports
Metadata
ExecuteGraph Attributes
Details
Best Practices
Compatibility
See also

### Short Description

ExecuteGraph runs graphs with user-specified settings and provides execution results and tracking details to output ports.

| Note
To be able to use this component, you need a separate jobflow license. Also, the component requires your project is executed on CloverETL Server.

Component | Same input metadata | Sorted inputs | Inputs | Outputs | Each to all outputs | Java | CTL | Auto-propagated metadata
--- | --- | --- | --- | --- | --- | --- | --- | ---
ExecuteGraph |  |  | 0-1 | 0-2 |  |  |  |

### Icon

### Ports

Port type | Number | Required | Description | Metadata
--- | --- | --- | --- | ---
Input | 0 |  | Input tokens with graph execution settings. | Any
Output | 0 |  | Execution information for successful graphs. | Any
1 |  | Execution information for unsuccessful graphs. | Any

The component can have single input port and two output ports attached.

### Metadata

ExecuteGraph does not propagate metadata from left to right or from right to left. It propagates metadata of its templates.

This component has metadata templates on all its ports. The templates are described in Details. See details on Metadata Templates.

### ExecuteGraph Attributes

Attribute | Req | Description | Possible values
--- | --- | --- | ---
Basic
Graph URL | yes | A path to executed graph. Only a single graph can be specified in this attribute. The value can be overridden by a value from input token, see Input mapping attribute. The graph referenced by this attribute is also used for all mapping dialogs - they display dictionary entries and tracking information based on this graph. |
Execution type | no | Specifies type of execution - synchronous (sequential) or asynchronous (parallel) execution model. Can be overridden by a value from input token, see Input mapping attribute. See Execution type. | synchronous (default) | asynchronous
Timeout | no | Maximal amount of time dedicated for graph run; by default in milliseconds, but other time units may be used. If the timeout interval elapses, the graph is aborted. The value can be overridden by a value from input token, see Input mapping attribute. Timeout attribute is ignored for asynchronous execution type. Use MonitorGraph component to watch the running graph. | 0 (unlimited) | positive number
Input mapping | no | Input mapping defines how data from incoming token overrides default execution settings. See Input mapping. | CTL transformation
Output mapping | no | Output mapping maps results of successful graphs to the first output port. See Output mapping. | CTL transformation
Error mapping | no | Error mapping maps results of unsuccessful graphs to the second output port. See Error mapping. | CTL transformation
Redirect error output | no | By default, results of failed graphs are sent to the second output port (error port). If this switch is true, results of unsuccessful graphs are sent to the first output port in the same way as successful graphs. | false (default) | true
Advanced
Auto-propagate graph parameters | no | Passes parameter values to executed graph (subgraph) without explicit mapping. If parameter exists in both jobflow and executed graph and if the parameter is public in the executed graph, the parameter value is passed to the executed graph. | No (default) | yes
Execution group | no | A name of execution group to which the executed graph belongs. An execution group can be used by KillGraph component as a named handler for a set of graphs to be interrupted. | string
Execution label | no | A text displayed in an execution view before graph name. | E.g. MyLabel
Cluster node ID | no | Cluster node ID which will be used for execution of graph. | string
Execute graph as daemon | no | By default, all graphs are executed in non-daemon mode, so none of them can live longer than the parent graph. Clover server automatically ensures for finished jobs that all non-daemon graphs are interrupted if they have not finished yet. If you want to start a graph which can live longer than parent graph, set this switch to true. | false (default) | true
Skip checkConfig | no | By default, the pre-execution configuration check of the graph is performed if and only if the check was performed on the parent job. This attribute can explicitly enable or disable the check. | boolean (default is inherited from parent job)
Stop processing on fail | no | By default, any failed graph causes the component stop executing other graphs; information about skipped tokens is sent to the error output port. This behaviour can be turned off by this attribute. | true (default) | false
Number of executors | no | By default, graphs executed in synchronous mode are triggered one after the other - the next graph is executed right the previous one finished. This option allows to increment number of simultaneously running graphs. Number of executors attribute defines how many graphs can be executed at once. All of them are monitored and once one of the running graphs finish processing another one is executed. This option is applied only to graphs executed in synchronous mode. | positive number (1 is default)
Locale | no | The default Locale of the executed graph. | locale ID, see List of all Locale
Time zone | no | The default Time Zone of the executed graph. | time zone ID, e.g. `America/New_York`

### Details

Execution type
Input mapping
Output mapping
Error mapping

The ExecuteGraph component runs a graph with specific settings, waits for the graph to finish, and monitors the graph results and tracking information.

The `.sgrf` files can be run by ExecuteGraph component as well as CloverETL graphs.

##### Execution Flow in Details

The component reads an input token, executes a graph based on incoming data values, waits for the graph to finish, and sends results to corresponding port. Results of successful graph are sent to the first output port. Results of failed graph are sent to second output port (error port).

If the graph run was successful, the component continues with processing of next input tokens. Otherwise, the component stops executing other graphs, and from now all incoming tokens are ignored, and information about ignored tokens is sent to error output port. The behaviour related to processing after first graph failure can be changed via attribute Stop processing on fail. If you set Stop processing on fail to false, the following graphs are executed.

##### Connected and Disconnected Ports

In case no input port is attached, only one graph is executed with default settings specified in the component's attributes.

In case the first output port is not connected, the component just prints out the graph results to the log.

In case the second output port (error port) is not attached, first failed graph causes interruption of the parent job. If you use Redirect error output, disconnected error port does not cause interruption of parent job.

##### Component Configuration

For graph execution, it is necessary to specify graph location. Optionally, you can set up execution type, initial values of graph parameters and dictionary content, timeout, execution group and several other attributes.

Most of execution settings can be specified on the component via component attributes described bellow. These settings could be considered as a default execution settings.

The default execution settings can be dynamically changed individually for each graph execution based on data from incoming token. Input mapping attribute is the place where this override is defined.

| Tip
Right-click the component and click Open Graph to access the graph that is executed. Similarly, in the ExecuteJobflow and ExecuteProfilerJob components, there are the Open Jobflow and Open Profiler Job options.

#### Execution type

The component supports synchronous (sequential) and asynchronous (parallel) graph execution.

- synchronous execution mode (default) - the component blocks until the graph has finished processing, so graphs are monitored by this component until the end of run.

- asynchronous execution mode - the component starts the graph and immediately sends the status information to the output. The graphs are just started by this component, the MonitorGraph component should be used for monitoring asynchronously executed graphs.

#### Input mapping

Input mapping attribute allows to override the settings of the component based on data from incoming token. Moreover, initial dictionary content and graph parameters of executed graph can be changed in input mapping.

Input mapping is a regular CTL transformation which is executed before each graph execution. Input token if any is only input for this mapping. The transformation's outputs consists of three records: RunConfig, JobParameters and Dictionary.

- RunConfig record represents execution settings. If a field of the record is not populated by this mapping, default value from respective attribute of the component is used instead.

  Field Name | Type | Description
  --- | --- | ---
  jobURL | string | Overrides component attribute Graph URL.
  executionType | string | Overrides component attribute Execution type.
  timeout | long | Overrides component attribute Timeout.
  executionGroup | string | Overrides component attribute Execution group.
  executionLabel | string | Overrides component attribute Execution label.
  clusterNodeId | string | Overrides component attribute Cluster node ID.
  daemon | boolean | Overrides component attribute Execute graph as daemon.
  skipCheckConfig | boolean | Overrides component attribute Skip checkConfig.
  locale | string | Overrides component attribute Locale.
  timeZone | string | Overrides component attribute Time zone.
  jobParameters | map[string, string] | graph parameters passed to the executed graph. Primary way of definition of graph parameters is direct mapping to JobParameters record available in input mapping dialog, where you can easily populate prepared graph parameters extracted from executed graph. Graph parameters defined via this map have the highest priority. This map can be used in case the set of graph parameters is not available in design time of jobflow.

- JobParameters record represents all internal and external graph parameters of triggered graph.

- Dictionary record represents input dictionary entries of triggered graph.

| Note
JobParameters and Dictionary records are available in transform dialog only if component attribute Graph URL links to an existing graph which is used as a template for extraction of graph parameters and dictionary structure. Only graph parameters and input dictionary entries from this graph can be populated by input mapping, no matter which graph will be actually executed in runtime.

#### Output mapping

Output mapping is a regular CTL transformation which is used to populate token passed to the first output port. The mapping is executed for successful graphs. Up to four input data records are available for this mapping.

If output mapping is empty, fields of RunStatus record are mapped to output by name.

- The input token configuring the graph execution. This record is not available for a component without input connector. This record has Port 0 displayed in the Type column.

  This is very helpful for passing through some fields from input token to output token.

- RunStatus record provides information about graph execution.

  Field Name | Type | Description
  --- | --- | ---
  runId | long | Unique identifier of graph run. In case of asynchronous execution type the value can be used for graph monitoring or interruption.
  orignalJobURL | string | Path to executed graph.
  startTime | long | Time of graph execution.
  endTime | long | Time of graph finish or null for asynchronous execution.
  duration | long | Graph execution time in milliseconds.
  executionGroup | string | Name of execution group to which the executed graph belongs.
  status | string | Final graph execution status (FINISHED_OK | ERROR | ABORTED | TIMEOUT | RUNNING for asynchronous execution).
  errException | string | Cause exception for failed graphs only.
  errMessage | string | Error message for failed graphs only.
  errComponent | string | Component ID which caused graph fail.
  errComponentType | string | Type of component which caused graph fail.

- Dictionary record provides content of output dictionary entries of the graph. This record is available for the mapping only if the attribute 'Graph URL' refers to a graph instance, which has an output dictionary entry.

- Tracking record provides tracking information usually available only in JMX interface of the running graph. This record is available for the mapping only if the Graph URL attribute refers to a graph instance.

  - Tracking fields available for whole graph:

    Field Name | Type | Description
    --- | --- | ---
    startTime | long | Time of graph execution.
    endTime | long | Time of graph finish or null for running graph.
    executionTime | long | Graph execution time in milliseconds.
    graphName | string | Name of executed graph.
    result | string | Graph execution status (FINISHED_OK | ERROR | ABORTED | TIMEOUT | RUNNING).
    runningPhase | integer | Index of running phase or null if graph is already finished.

  - Tracking fields available for a graph phase:

    Field Name | Type | Description
    --- | --- | ---
    startTime | date | Time of phase execution.
    endTime | date | Time of phase finish or null for running phase.
    executionTime | long | Phase execution time in milliseconds.
    memoryUtilization | long | Graph memory footprint (in bytes).
    result | string | Phase execution status (FINISHED_OK | ERROR | ABORTED | RUNNING).

  - Tracking fields available for a component:

    Field Name | Type | Description
    --- | --- | ---
    name | string | Name of the component.
    usageCPU | number | Actual CPU time used by the component expressed by number from interval (0, 1) (0 means 0% of CPU is used by the component, 1 means 100% of CPU is used by the component).
    usageUser | number | Actual CPU time used by the component in user mode expressed by number from interval (0, 1) (0 means 0% of CPU is used by the component, 1 means 100% of CPU is used by the component).
    peakUsageCPU | number | Maximal CPU time used by the component expressed by number from interval (0, 1) (0 means 0% of CPU is used by the component, 1 means 100% of CPU is used by the component).
    peakUsageUser | number | Maximal CPU time used by the component in user mode expressed by number from interval (0, 1) (0 means 0% of CPU is used by the component, 1 means 100% of CPU is used by the component).
    totalCPUTime | long | Number of milliseconds of CPU time used by this component.
    totalUserTime | long | Number of milliseconds of CPU time in user mode used by this component.
    result | string | Component execution status (FINISHED_OK | ERROR | ABORTED | RUNNING).

  - Tracking fields available for an input or output port:

    Field Name | Type | Description
    --- | --- | ---
    byteFlow | integer | Number of bytes passed through this port per seconds.
    bytePeak | integer | Maximal byteFlow registered on this port.
    totalBytes | long | Number of bytes passed through this port.
    recordFlow | integer | Number of records passed through this port per seconds.
    recordPeak | integer | Maximal recordFlow registered on this port.
    totalRecords | long | Number of records passed through this port.
    waitingRecords | integer | Number of records cached on the edge connected to the port.
    averageWaitingRecords | integer | Average number of records cached on the edge connected to the port.
    usedMemory | integer | Memory footprint (in bytes) of the attached edge.

#### Error mapping

Error mapping is a regular CTL transformation. The field are the same as in the output mapping.

The error mapping is used if and only if the graph finished unsuccessfully and the second output port is populated instead of the first one.

If error mapping is empty, fields of RunStatus record are mapped to output by name.

### Best Practices

| Tip
If you drag a `*.grf` file and drop it into the jobflow pane, you will add the ExecuteGraph component.

### Compatibility

Since CloverETL 4.0.0-M2 you can set up attribute Execution label.

Since CloverETL 4.1.0 you can auto-propagate graph parameters.

Since CloverETL 4.1.0 empty output mapping and error mapping work as mapping fields by name: `$out.0.* = $in.0.*; $out.0.* = $in.1.*;`

### See also

RunGraph
Common Properties of Components
Specific Attribute Types
Common Properties of Job Control
Job control Comparison

Prev | Up | Next
Condition | Home | Table of Contents | ExecuteJobflow