Component Allocation

This attribute is taken into account only on CloverETL Cluster environment.

Allocation attribute is common for all ETL components. This attribute is used for cluster graph processing to plan how many instances of the component will be executed and on which cluster nodes. Allocation is our basic concept for parallelization of data processing and inter-cluster-node data routing.

Allocation can be specified in three different ways:

Component allocation dialog

Figure 30.11. Component allocation dialog


Allocation is automatically inherited from neighboring components. So, continuous graph may have only single component with an allocation and this allocation is used by all other components as well. All components of clustered graphs are decorated by number of instances (x3) in which the component will be finally executed - so called allocation cardinality. This annotations are updated on graph save operation. Allocation cardinality derived from neighbors are drawn with gray italic font and the cardinality derived from allocation defined right on the component is printed out with a solid font.

Allocation cardinality decorator

Figure 30.12. Allocation cardinality decorator


Two interconnected components have to have compatible allocations - number of specified workers has to be equal. Only exception from this rule are cluster components, which are dedicated just to change level of parallelism. Cluster partitioners change single-worker allocation to multi-worker allocation. On the other hand cluster gathers change multi-worker allocation to single-worker allocation.

More details about clustered graph processing are available in CloverETL Server Documentation in Cluster part.