Jobflow Component

If you get file URLs from the input port having Stop processing on fail set to true and error occurs, the record causing error and the following ones are skipped.

Copy .txt files from ${DATAIN_DIR}/FO/ to ${DATAOUT_DIR}/FO/. The target directory may not exist. If the files in the target directory exist, they should be overwritten.

Solution

Use the attributes Source file URL, Target file URL and Create parent directories.

Copy files f1.txt f2.txt and f3.txt. If you cannot copy a file, continue with the following one.

-rw-rw-r--. 1 clover clover 0 Feb 23 18:11 f1.txt
-rw-------. 1 user23 user23 0 Feb 23 18:11 f2.txt
-rw-rw-r--. 1 clover clover 0 Feb 23 18:11 f3.txt

Solution with Source File URL attribute

The solution without reading file URLs from an input edge is same as in the previous example. Use the attributes Source file URL, Target file URL and Create parent directories.

Note that only files f1.txt and f2.txt will be copied. The file f2.txt cannot be copied as the user 'clover' does not have read access to the file.

Solution with Input Edge

If you read URLs of files from an input edge one by one, use the attributes Target file URL, Create parent directories Input mapping and Stop processing on fail.

In Input mapping, you map Source URL.

Note that you need to uncheck the attribute Stop processing on fail. Otherwise only the file f1.txt would be copied. (Assuming that the files come in ascending order.)

This example shows non-recursive copying of files from one directory to another.

Copy files and directories from ${DATAIN_DIR}/abc to ${DATAOUT_DIR}/def. If a directory in ${DATAIN_DIR}/abc contains a file or a directory, these files and directories should not be copied.

Source directory

${DATAIN_DIR}/abc/file.txt
${DATAIN_DIR}/abc/dirA/file2.txt

Expected result

${DATAIN_DIR}/def/file.txt
${DATAIN_DIR}/def/dirA

Solution

Use ListFiles to list the files and directories to be copied. In ListFiles set File URL attribute to the source directory.

Attribute	Value
File URL	${DATAIN_DIR}/abc/

In CopyFiles, copy the files and directories from the list received from an input edge.

Attribute	Value
Target file URL	${DATAIN_DIR}/def/
Create parent directories	true
Input mapping	//#CTL2 // Transforms input record into output record. function integer transform() { $out.0.sourceURL = $in.0.URL; return ALL; }

	Note
	The ListFiles component is necessary to list all files and directories. If you non-recursively copied the files with CopyFiles only, the copying would stop when a first subdirectory is reached.