Data Policy affects processing (parsing) of incorrect or incomplete records. This can be specified by the Data Policy attribute. There are three options:
Strict. This data policy is set by default. It means that data parsing stops if a record field with an incorrect value or format is read. Next processing is aborted.
Controlled.
This data policy means that every error is logged, but incorrect records are skipped and data parsing continues.
Generally, incorrect records with error information are logged into stdout.
Only
ParallelReader,
FlatFileReader,
JSONReader
and SpreadsheetDataReader
enable to sent them out through the optional second port.
See error metadata description for particular above mentioned readers.
Lenient. This data policy means that incorrect records are only skipped and data parsing continues.
Data policy can be set in the following Readers:
| ComplexDataReader |
| DBFDataReader |
| DBInputTable |
| FlatFileReader |
| JSONReader |
| MultiLevelReader |
| ParallelReader |
| SpreadsheetDataReader |
| XMLReader |
| XMLXPathReader |