Home | Table of Contents | CloverDataReader | CloverETL 4.6.1
--- | --- | ---
Prev | Readers | Next

## CloverDataReader

# Not available in Community Designer

### Short Description

CloverDataReader reads data stored in our internal binary Clover data format files. It can also read data from compressed files, or dictionary.

Component | Data source | Input ports | Output ports | Each to all outputs | Different to different outputs | Transformation | Transf. req. | Java | CTL | Auto-propagated metadata
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
CloverDataReader | clover binary file | 1 | 1-n |  |  |  |  |  |  |

### Icon

### Ports

Port type | Number | Required | Description | Metadata
--- | --- | --- | --- | ---
Input | 0 |  | For Input Port Reading correct data records | Include specific `byte`/`cbyte` field
Output | 0 |  | For correct data records | Any
1-n |  | For correct data records | Output 0

### Metadata

CloverDataReader does not propagate metadata.

CloverDataReader has no metadata template, but it can extract metadata from clover file and propagate it forward as it would have a template. (Available since 4.1.0-M1)

Metadata on the input port has to include a `byte` or `cbyte` or `string` field.

Metadata on output port has to be the same as metadata of data from the file.

Metadata can use Autofilling Functions.

### CloverDataReader Attributes

Attribute | Req | Description | Possible values
--- | --- | --- | ---
Basic
File URL | yes | Attribute specifying what data source(s) will be read (Clover data file, input port, dictionary). See Supported File URL Formats for Readers. |
Advanced
Number of skipped records |  | Number of records to be skipped. See Selecting Input Records. | 0-N
Max number of records |  | Maximum number of records to be read. See Selecting Input Records. | 0-N
Number of skipped records per source |  | Skip the first `n` records of each file. | 0-N
Max number of records per source |  | Reads maximally `n` records from each file. | 0-N
Deprecated
Index file URL |  | Name of the index file, including path. If not specified, all records are read. |
Start record |  | Has exclusive meaning: Last record before the first that is already read. Has lower priority than Number of skipped records. | 0 (default) | 1-n
Final record |  | Has inclusive meaning: Last record to be read. Has lower priority than Max number of records. | all (default) | 1-n

### Examples

Reading a Clover Data File
Omitting Leading Records
Omitting Leading Records of Each File
Reading at most n Records in Total
Reading at most n Records per File

#### Reading a Clover Data File

Read all records from the Clover data file.

##### Solution

Set up the File URL attribute.

Attribute | Value
--- | ---
File URL | ${DATAIN_DIR}/my-clover-file.cdf

CloverDataReader will read all the records from the file(s).

#### Omitting Leading Records

You have two Clover data files. First 3 records contain uninteresting data and should not be read. The uninteresting records are in the first file. (The records have been sorted and partitioned for example.)

`greengrocers1.cdf`

    bread
    honey
    raisins
    pears
    plums

`greengrocers2.cdf`

    carrot
    peas
    radish

##### Solution

Set up File URL and Number of skipped records attributes.

Attribute | Value
--- | ---
File URL | ${DATAIN_DIR}/greengrocers1.cdf;${DATAIN_DIR}/greengrocers2.cdf
Number of skipped records | 3

CloverDataReader reads the following items:

    pears
    plums
    carrot
    peas
    radish

#### Omitting Leading Records of Each File

There are two Clover data files: `list1.cdf` and `list2.cdf`. Each file starts with one record to be omitted.

`list1.cdf`

    Goods
    cardigan
    shirt
    trousers

`list2.cdf`

    Goods
    shoes
    sox

##### Solution

Set up File URL and Number of skipped records per source attributes.

Attribute | Value
--- | ---
File URL | ${DATAIN_DIR}/list1.cdf;${DATAIN_DIR}/list2.cdf
Number of skipped records per source | 1

CloverDataReader sends the following records to the output:

    cardigan
    shirt
    trousers
    shoes
    sox

#### Reading at most n Records in Total

You have three files `stationery1.cdf`, `stationery2.cdf` and `stationery3.cdf` and you need to read six records in total from all files.

`stationery1.cdf`

    pen
    pencil
    marker
    paintbrush

`stationery2.cdf`

    ink
    water colors
    oil colors

`stationery3.cdf`

    notebook
    coloring book

##### Solution

Set up File URL and Max number of records attributes.

Attribute | Value
--- | ---
File URL | ${DATAIN_DIR}/stationery1.cdf;${DATAIN_DIR}/stationery2.cdf;${DATAIN_DIR}/stationery3.cdf
Max number of records | 6

CloverDataReader sends all 4 records from `stationery1.cdf` and 2 of 3 records from `stationery2.cdf` to the output port. No record from the file `stationery3.cdf` is sent the output port as the limit has been reached already.

    pen
    pencil
    marker
    paintbrush
    ink
    water colors

#### Reading at most n Records per File

You have three Clover data files (`stationery[1-3].cdf`) from the previous example. Read at most 3 records from each file.

##### Solution

Set up File URL and Max number of records per source attributes.

Attribute | Value
--- | ---
File URL | ${DATAIN_DIR}/stationery1.cdf;${DATAIN_DIR}/stationery2.cdf;${DATAIN_DIR}/stationery3.cdf
Max number of records per source | 3

CloverDataReader reads 3 records from `stationery1.cdf`, 3 records from `stationery2.cdf` and 2 of 2 records from `stationery3.cdf`.

    pen
    pencil
    marker
    ink
    water colors
    oil colors
    notebook
    coloring book

### Compatibility

#### 2.9

Since 2.9 version of CloverETL, CloverDataWriter also writes a header to output files with the version number. For this reason, CloverDataReader expects that files in Clover binary format contain such a header with the version number. CloverDataReader 2.9 cannot read files written by older versions of CloverETL nor these older versions can read data written by CloverDataWriter 2.9.

#### 4.0

The internal structure of the zip archive has changed. Graphs that rely on the structure will stop working. Graphs that use plain zip file URL without internal entry specification are not affected: CloverDataReader with File URL `zip:(${DATAOUT_DIR}/employees.zip)#DATA/employees` will need to be fixed to use `zip:(${DATAOUT_DIR}/employees.zip)` instead.

#### 4.1.0-M1

Since 4.1.0-M1, CloverDataReader can extract metadata template from clover file. It can be seen as a metadata template corresponding to the file.

#### 4.4.0-M2

Since 4.4.0-M2, CloverDataReader can read from input port just from `byte` or `cbyte` field.

### See also

CloverDataWriter
Common Properties of Components
Specific Attribute Types
Common Properties of Readers
Readers Comparison

Prev | Up | Next
Java Interfaces for Readers | Home | Table of Contents | ComplexDataReader