Chapter 18. Engine Configuration
CloverETL internal settings (defaults) are stored in defaultProperties
file located in the CloverETL engine (plugins/com.cloveretl.gui/lib/lib/cloveretl.engine.jar)
in its org/jetel/data subfolder. This source file contains various parameters
that are loaded at run-time and used during the transformation execution.
If you modify the values right in the defaultProperties file,
such change will be applied for all graph runs.
To change the values just for the current graph(s), create a local file with only those properties you need to override.
Place the file in the project directory.
To instruct CloverETL
to retrieve the properties from this local file, use the -Dclover.engine.config.file=/full/path/to/file.properties VM parameter
of ETL Runtime or define a path to the file in CloverETL Runtime configuration.
Here we present some of the properties and their values as they are presented in the
defaultProperties file:
Record.RECORD_LIMIT_SIZE = 33554432It limits the maximum size of a record. Theoretically, the limit is tens of MBs, but you should keep it as low as possible for an easier error detection. See Edge Memory Allocation for more details on memory demands.
Record.FIELD_LIMIT_SIZE = 33554432It limits the maximum size of one field within a record. See Edge Memory Allocation for more details on memory demands.
Record.RECORD_INITIAL_SIZE = 65536It sets the initial amount of memory allocated to each record. The memory can grow dynamically up to
Record.RECORD_LIMIT_SIZE, depending on how memory-greedy an edge is. See Edge Memory Allocation.
Record.FIELD_INITIAL_SIZE = 65536Sets the initial amount of memory allocated to each field within a record. The memory can grow dynamically up to
Record.FIELD_LIMIT_SIZE, depending on how memory-greedy an edge is. See Edge Memory Allocation.
Record.DEFAULT_COMPRESSION_LEVEL = 5This sets the compression level for compressed data fields (
cbyte).
DEFAULT_INTERNAL_IO_BUFFER_SIZE = 32768It determines the internal buffer size the components allocate for I/O operations. Increasing this value affects performance negligibly.
USE_DIRECT_MEMORY = trueThe clover engine intensively uses direct memory for data records manipulation. For example underlying memory of CloverBuffer (container for serialized data records) is allocated outside of the Java heap space in direct memory. This attribute is by default
truein order to improve performance. However, direct memory is out of control of a java virtual machine, so try to turn off usage of direct memory in case an OutOfMemory exception occurs.
DEFAULT_DATE_FORMAT = yyyy-MM-dd
DEFAULT_TIME_FORMAT = HH:mm:ss
DEFAULT_DATETIME_FORMAT = yyyy-MM-dd HH:mm:ss
DEFAULT_REGEXP_TRUE_STRING = true|T|TRUE|YES|Y|t|1|yes|y
DEFAULT_REGEXP_FALSE_STRING = false|F|FALSE|NO|N|f|0|no|n
DataParser.DEFAULT_CHARSET_DECODER = UTF-8
DataFormatter.DEFAULT_CHARSET_ENCODER = UTF-8
Lookup.LOOKUP_INITIAL_CAPACITY = 512The initial capacity of a lookup table when created without specifying the size.
DataFieldMetadata.DECIMAL_LENGTH = 12It determines the default maximum precision of decimal data field metadata. Precision is the number of digits in a number, e.g., the number 123.45 has a precision of 5.
DataFieldMetadata.DECIMAL_SCALE = 2It determines the default scale of decimal data field metadata. Scale is the number of digits to the right of the decimal point in a number, e.g., the number 123.45 has a scale of 2.
Record.MAX_RECORD_SIZE = 33554432![[Note]](figures/note.png)
Note This is a deprecated property. Nowadays, you should use
Record.RECORD_LIMIT_SIZE.It limits the maximum size of a record. Theoretically, the limit is tens of MBs, but you should keep it as low as possible for an easier error detection.
![[Important]](figures/important.png) | Important |
|---|---|
You can define locale that should be used as the default one. The setting is the following:
By default, system locale is used by CloverETL.
If you uncomment this row you can set the
Similarly, the default time zone can be overridden by uncommenting the following entry:
See the Time Zone section for more information about time zones. |
| Compatibility |
|---|---|
In 4.4.0-M2, the default encoding was changed from ISO-8859-1 to UTF-8.
Therefore |