Available in Community Designer

Editing the Input mapping attribute opens the Transform Editor where you can decide which component attributes should be set using input record.

Figure 61.4. Transform Editor in HTTPConnector

The dialog provides you with all the power and features known from Transform Editor and CTL.

	Note
	All kinds of CTL functions are available to modify the input field value to be used.

Since release 3.5.4 you can set up multipart entities in transform editor. Input mapping now offers new fields derived from value of Multipart entities attribute. For example field1;field2 as value of multipart entities generates following fields.

Figure 61.5. Multipart entities in input mapping

The generated fields can be used to control multipart entities.

If you deal with Multipart entities, you have to use POST method.

function integer transform() {
 	$out.4.field1_EntityContent="My custom content";
 	$out.4.field1_EntityFileNameAttribute="MyFilename";
 	returnALL;
}

­­CB5PZVJDq5RyTWoZqxvtjlbVM0CrMa3Mt 
Content­Disposition: form­data; name="field1"; filename="MyFilename" 
Content­Type: text/plain; charset=UTF­8 
Content­Transfer­Encoding: 8bit 
 
My custom content 
­­CB5PZVJDq5RyTWoZqxvtjlbVM0CrMa3Mt 

$out.3.field3_EntitySourceFile = "${PROJECT}/workspace.prm";

­­3xEKe3wUSOl2cRnjwh1UsPVnDOoL7D 
Content­Disposition: form­data; name="field3"; filename="workspace.prm" 
Content­Type: application/octet­stream 
Content­Transfer­Encoding: binary 
 
... [here is content of file] 
 
­­3xEKe3wUSOl2cRnjwh1UsPVnDOoL7D­­

Editing the attribute opens the Transform Editor where you can decide what should be sent to an output port.

Figure 61.6. Transform Editor in HTTPConnector

The dialog provides you with all the power and features known from Transform Editor and CTL.

To do the mapping in a few basic steps:

You can map various data to output port:

	Note
	Output mapping uses CTL (you can switch to the Source tab). All kinds of functions are available to modify the value to be stored in the output field. $out.0.prices = find($in.1.content, "price: .*? USD") finds all occurrences of the form price: [some text] USD in the response content.

If you let output mapping empty, the default output mapping is used:

$out.0.* = $in.0.*;
$out.0.* = $in.1.*;

The default mapping has been introduced in version 4.1.0.

Editing the Error mapping attribute opens the Transform Editor where you can map error details to an output port. The behavior is very similar to the Output mapping

If you let error mapping empty, the default error mapping is used:

$out.1.* = $in.0.*;
$out.1.* = $in.1.*;

The default mapping has been introduced in version 4.1.0.

When the graph's log level is set to DEBUG, the HTTPConnector prints the HTTP request and response to graph log.

Download a content of a web page www.cloveretl.com using HTTPConnector. Save the result to the file for further processing.

Solution

Use URL and Output file URL attributes. The downloaded page will be saved into result.html file in ${DATAOUT_DIR} directory.

Download document from https://protected.example.org/document.html. The site requires HTTP basic authentication.

Solution

Set up URL, Output file URL, Username and Password attributes. We suggest to use secure parameters to store your password.

Alternative solution is to connect an edge to the first output port instead of filling Output file URL attribute. The result will be send to the edge. No output mapping is necessary.

Download content of page http://www.cloveretl.com/. The page is accessible via proxy on 10.0.3.5 listening on tcp port 3128.

Solution

Use the URL attribute. You can use Output file URL to write result to the file or connect an output edge.

Note: The proxy may introduce some limitations. E.g. It may deny you to connect via HTTPS etc.

The problem to be solved is the similar to the previous example. The difference is, that proxy requires username (test) and password (securePassword).

Solution

Connect to Twitter API and get some tweets about java.

Solution

Use URL, OAuth Consumer key, OAuth Consumer secret, OAuth Access Token and OAuth Access Token secret attributes.

Connect an edge to the first output port to pass results by egde or fill in Output file URL attribute to write down results to a file.

Note: The credentials in this example are not valid, you have to use your own credentials.

Send a file using multipart entities. The file content is available in field1 field.

Solution

Use URL, Request method, Multipart entities and Input mapping attributes.

function integer transform() {
 	$out.4.field1_EntityContent = $in.0.field1;

 	return ALL;
}

Map multipart entities in the Input mapping dialog.

Connect to www.my-sometimes-responding-server.com which sometimes fails to respond. The response has to be returned within 20 seconds, otherwise connection should be considered as nonresponding. Make at most 5 attempts in total.

Solution

Use Timeout to set up time limit on connection to avoid waiting if server does not reply. If server responds sometimes only, use Retry count to ask several times.

Timeout is in milliseconds. 20000 milliseconds is 20 seconds. Retry count set to 4 causes up to 4 additional retries (if necessary). At most five requests are performed in total.