Short Description |
Ports |
Metadata |
SalesforceReader Attributes |
Details |
Examples |
Compatibility |
See also |
SalesforceReader reads records from Salesforce using SOAP API.
Which Salesforce reader? | |
---|---|
If you need to read a small number of records, read attachments, or use functions and subqueries, use SalesforceReader. If you need to read a large number of records, use SalesforceBulkReader. |
Component | Data source | Input ports | Output ports | Each to all outputs | Different to different outputs | Transformation | Transf. req. | Java | CTL | Auto-propagated metadata |
---|---|---|---|---|---|---|---|---|---|---|
SalesforceReader | database | 0 | 1 |
Port type | Number | Required | Description | Metadata |
---|---|---|---|---|
Output | 0 | SOQL query results | output0 |
SalesforceReader does not propagate metadata.
SalesforceReader has no metadata templates.
SalesforceReader has no special requirements on metadata names or field data types.
Attribute | Req | Description | Possible values |
---|---|---|---|
Basic | |||
Connection | yes | A Salesforce connection. See Salesforce connection. | e.g. MySFConnection |
SOQL query | yes |
A query for retrieving data from Salesforce. The component allows you to use subset of SOQL language. If you query the records, you should use API names for objects and fields. The API name can differ from the name of an object in Salesforce web gui. You can use graph parameters in SOQL query. | e.g. SELECT Name, Website FROM Account WHERE Industry = 'Energy' |
Output mapping | Mapping from query fields to output metadata fields | Map by name (default) | |
Read mode | Defines in what way are records read from Salesforce. You can enable reading deleted records, and then such records will be returned by the component. To distinguish deleted and not deleted records, query the isDeleted field. See Reading deleted records details for more details and Reading deleted records for an example. | Do not return deleted or archived records (default) | Return also deleted or archived records |
SalesforceReader reads records from Salesforce using SOAP API, which performs the operations synchronously making it suitable for retrieving small datasets.
To use the component, create a Salesforce connection, enter SOQL query, and specify the output mapping. If you perform the steps in this order, the transform editor can provide you with metadata extracted from the SOQL query. Therefore, you will be able to map the fields with drag and drop.
SalesforceReader uses Salesforce Object Query Language (SOQL) to query data in Salesforce.
Due to different implementation of almost each function in API, we support the following functions.
Aggregate Functions |
Date functions |
Misc functions |
The table shows returned data type for particular argument data type.
Function | Parameter type | ||||
---|---|---|---|---|---|
Boolen | Decimal | Integer | Date | String | |
avg() | - | Decimal[a] | Decimal[a] | - | - |
count() | - | Integer | Integer | Integer | Integer |
count_distinct() | no | Integer | Integer | Integer | Integer |
min() | - | Decimal[b] | Integer | Date | String |
max() | - | Decimal[b] | Integer | Date | String |
sum() | - | Decimal[c] | Integer | - | - |
[a] Decimal with 'maximal' precision and scale is used (decimal[64, 32]) [b] Precision and scale are derived from the first function parameter. [c] Scale is derived from the first function parameter and precision is fixed to 64. |
See also Salesforce documentation on aggregate functions: https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_agg_functions.htm.
Function | Result type | Note |
---|---|---|
CALENDAR_MONTH() | Integer | |
CALENDAR_QUARTER() | Integer | |
CALENDAR_YEAR() | Integer | |
DAY_IN_MONTH() | Integer | |
DAY_IN_WEEK() | Integer | |
DAY_IN_YEAR() | Integer | |
DAY_ONLY() | Date | |
FICSAL_MONTH() | Integer | |
FISCAL_QUARTER() | Integer | |
FISCAL_YEAR() | Integer | |
HOUR_IN_DAY() | Integer | |
WEEK_IN_MONTH() | Integer | |
WEEK_IN_YEAR() | Integer | |
convertTimezone() | Date |
Must be used in a date function.
For example
See also documentation on |
See also Salesforce documentation on date functions https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_date_functions.htm.
Function | Result type | Note |
---|---|---|
convertCurrency() | Decimal [a] | https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_sosl_querying_currency_fields.htm |
format() | String | https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_sosl_format.htm |
distance() | Decimal [b] | https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_geolocate.htm |
toLabel() | String | https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_tolabel.htm |
grouping() | Integer | https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_groupby_grouping.htm |
[a] decimal[64, 6] [b] Decimal with 'maximal' precision and scale is used (decimal[64, 32])) |
When the SOQL query contains a subquery, LEFT OUTER JOIN is performed to provide flat data structure.
If no records have been returned by the subquery, null
values are returned for subquery fields.
For example, if you select opportunities in a subquery and an account from root query has no opportunity, a single
record with null
subquery fields is returned for this account. If the account has multiple
opportunities, multiple records are produced.
The output records come out in arbitrary order unless you use ORDER BY in your query.
If you read less than 10-15k of records, it is generally better to use SOAP API because it will use less API requests.
Compound fields group together several fields to represent a complex data type, such as addresses or locations. SalesforceReader does not support reading compound fields as a whole, you need to read their separate parts. The dialog for defining SOQL shows the individual fields of compound fields, e.g. BillingStreet, BillingCity, etc.
For example, to read BillingAddress it is not possible to use the following SOQL:
SELECT BillingAddress FROM Account
Instead you need to read its individual component fields:
SELECT BillingStreet, BillingCity, BillingPostalCode FROM Account
See https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/compound_fields.htm
In subqueries, you cannot read data from binary fields or use aggregate functions. This is limitation of the SOAP API.
When a Salesforce record is deleted it is first moved to Recycle Bin, from where it can be restored. Then automatically after a time interval, or after manually emptying the Recycle Bin, the record will be marked to be permanently deleted. Records marked to be permanently deleted cannot be restored, and are wiped from the database automatically by a background process.
If a record is deleted using the Hard Delete operation, it skips the Recycle Bin and is immediately ready to be permanently deleted.
Reading deleted records returns records which are not wiped yet, ie. also those that are not in the Recycle Bin anymore but were not permanently deleted yet.
SalesforceReader uses multiple API calls during its run. All of them count towards your Salesforce API request limit. The precise call flow is:
Login
Extract metadata of expected result of query. The number of requests depends on complexity of query. In general, every unique Salesforce object used in the query will result in 1 API request.
Send the query and iterate over the result. An API request must be sent for every 2000 returned records, so the number of requests depends on the number of records in the result. This is a limitation of the SOAP API.
Reading records from Salesforce |
Query on multiple objects (tables) |
Query on multiple objects (tables) II |
Using aggregate functions |
Reading deleted records |
This example shows the basic functionality of SalesforceReader.
Select FirstName, LastName and Email from Contact.
Create a Salesforce connection.
In SalesforceReader, set up the Connection, SOQL query and Output mapping parameters.
Attribute | Value |
---|---|
Connection | Connection from the first step |
SOQL query | SELECT FirstName, LastName, Email FROM Contact |
Output mapping |
//#CTL2 function integer transform() { $out.0.Email = $in.0.Email; $out.0.FirstName = $in.0.FirstName; $out.0.LastName = $in.0.LastName; return ALL; } |
This example shows the way to query records from two objects (tables) with a parent-child relationship. The solution of this example and the following ones shows only SOQL query as the rest of the configuration has already been shown in the first example.
Select account name and corresponding parent account name from Account.
SELECT Name, Parent.Name FROM Account
Select account name and details of corresponding opportunities: name, type, amount, description.
SELECT Account.Name, (SELECT Opportunity.Name, Opportunity.Type, Opportunity.Amount, Opportunity.Description FROM Opportunities) FROM Account
This example shows the way to use Salesforce aggregate functions.
Select account names and for each account count sum from opportunities we won. The result should be sorted according to the sum in descending order.
SELECT Account.Name, SUM(Amount) FROM Opportunity WHERE IsWon = true GROUP BY Account.Name ORDER BY SUM(Amount) DESC
This example shows how to read deleted records. See Reading deleted records details for more details on how records are deleted in Salesforce. Reading deleted records can be used for backup or for synchronisation with a data warehouse where you need to mirror the delete operations.
Set the SalesforceReader attribute Read mode to Return also deleted or archived records.
Read the records and their IsDeleted field to distinguish deleted and not deleted records:
SELECT Name, IsDeleted FROM Account
SalesforceReader is available since 4.4.0-M2. It uses Salesforce SOAP API version 37.0.
Since CloverETL 4.5.0-M2, SalesforceReader uses Salesforce SOAP API version 39.0.