LibraryToggle FramesPrintFeedback

The sql: component allows you to work with databases using JDBC queries. The difference between this component and JDBC component is that in case of SQL the query is a property of the endpoint and it uses message payload as parameters passed to the query.

This component uses spring-jdbc behind the scenes for the actual SQL handling.

The SQL component also supports:

  • a JDBC based repository for the Idempotent Consumer EIP pattern. See further below.

  • a JDBC based repository for the Aggregator EIP pattern. See further below.

[Important]Important

This component can be used as a Transactional Client.

Option Type Default Description
dataSourceRef String null Fuse Mediation Router 1.5.1/2.0: Reference to a DataSource to look up in the registry.
placeholder String # Camel 2.4: Specifies a character that will be replaced to ? in SQL query. Notice, that it is simple String.replaceAll() operation and no SQL parsing is involved (quoted strings will also change)
template.<xxx> null Sets additional options on the Spring JdbcTemplate that is used behind the scenes to execute the queries. For instance, template.maxRows=10. For detailed documentation, see the JdbcTemplate javadoc documentation.

When performing update operations, the SQL Component stores the update count in the following message headers:

Header Description
SqlProducer.UPDATE_COUNT Fuse Mediation Router 1.x: The number of rows updated for update operations, returned as an Integer object.
CamelSqlUpdateCount Fuse Mediation Router 2.0: The number of rows updated for update operations, returned as an Integer object.
CamelSqlRowCount Fuse Mediation Router 2.0: The number of rows returned for select operations, returned as an Integer object.
CamelSqlQuery Camel 2.8: Query to execute. This query takes precedence over the query specified in the endpoint URI. Note that query parameters in the header are represented by a ? instead of a # symbol

In the sample below we execute a query and retrieve the result as a List of rows, where each row is a Map<String, Object and the key is the column name.

First, we set up a table to use for our sample. As this is based on an unit test, we do it java code:

// this is the database we create with some initial data for our unit test
jdbcTemplate.execute("create table projects (id integer primary key,"
                     + "project varchar(10), license varchar(5))");
jdbcTemplate.execute("insert into projects values (1, 'Camel', 'ASF')");
jdbcTemplate.execute("insert into projects values (2, 'AMQ', 'ASF')");
jdbcTemplate.execute("insert into projects values (3, 'Linux', 'XXX')");

Then we configure our route and our sql component. Notice that we use a direct endpoint in front of the sql endpoint. This allows us to send an exchange to the direct endpoint with the URI, direct:simple, which is much easier for the client to use than the long sql: URI. Note that the DataSource is looked up up in the registry, so we can use standard Spring XML to configure our DataSource.

from("direct:simple")
    .to("sql:select * from projects where license = # order by id?dataSourceRef=jdbc/myDataSource")
    .to("mock:result");

And then we fire the message into the direct endpoint that will route it to our sql component that queries the database.

MockEndpoint mock = getMockEndpoint("mock:result");
mock.expectedMessageCount(1);

// send the query to direct that will route it to the sql where we will execute the query
// and bind the parameters with the data from the body. The body only contains one value
// in this case (XXX) but if we should use multi values then the body will be iterated
// so we could supply a List<String> instead containing each binding value.
template.sendBody("direct:simple", "XXX");

mock.assertIsSatisfied();

// the result is a List
List received = assertIsInstanceOf(List.class, mock.getReceivedExchanges().get(0).getIn().getBody());

// and each row in the list is a Map
Map row = assertIsInstanceOf(Map.class, received.get(0));

// and we should be able the get the project from the map that should be Linux
assertEquals("Linux", row.get("PROJECT"));

We could configure the DataSource in Spring XML as follows:

 <jee:jndi-lookup id="myDS" jndi-name="jdbc/myDataSource"/> 

In this section we will use the JDBC based idempotent repository.

First we have to create the database table which will be used by the idempotent repository.

For Camel 2.7, we use the follwing schema:

 CREATE TABLE CAMEL_MESSAGEPROCESSED (
   processorName VARCHAR(255),
   messageId VARCHAR(100)
 )

In Camel 2.8, we added the createdAt column:

 CREATE TABLE CAMEL_MESSAGEPROCESSED (
   processorName VARCHAR(255),
   messageId VARCHAR(100),
   createdAt TIMESTAMP
 )

We recommend to have a unique constraint on the columns processorName and messageId. Because the syntax for this constraint differs for database to database, we do not show it here.

Second we need to setup a javax.sql.DataSource in the spring XML file:

<bean id="dataSource" class="org.springframework.jdbc.datasource.SingleConnectionDataSource">
    <property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
    <property name="url" value="jdbc:hsqldb:mem:camel_jdbc"/>
    <property name="username" value="sa"/>
    <property name="password" value=""/>
</bean>

And finally we can create our JDBC idempotent repository in the spring XML file as well:

<bean id="messageIdRepository" class="org.apache.camel.processor.idempotent.jdbc.JdbcMessageIdRepository">
	<constructor-arg ref="dataSource" />
	<constructor-arg value="myProcessorName" />
</bean>

<camel:camelContext>
	<camel:errorHandler id="deadLetterChannel" type="DeadLetterChannel" deadLetterUri="mock:error">
		<camel:redeliveryPolicy maximumRedeliveries="0" maximumRedeliveryDelay="0" logStackTrace="false" />
	</camel:errorHandler>
	
	<camel:route id="JdbcMessageIdRepositoryTest" errorHandlerRef="deadLetterChannel">
		<camel:from uri="direct:start" />
		<camel:idempotentConsumer messageIdRepositoryRef="messageIdRepository">
			<camel:header>messageId</camel:header>
			<camel:to uri="mock:result" />
		</camel:idempotentConsumer>
	</camel:route>
</camel:camelContext>

Available as of Camel 2.6

[Important]Using JdbcAggregationRepository in Camel 2.6

In Camel 2.6, the JdbcAggregationRepository is provided in the camel-jdbc-aggregator component. From Camel 2.7 onwards, the JdbcAggregationRepository is provided in the camel-sql component.

JdbcAggregationRepository is an AggregationRepository which on the fly persists the aggregated messages. This ensures that you will not loose messages, as the default aggregator will use an in memory only AggregationRepository. The JdbcAggregationRepository allows together with Camel to provide persistent support for the Aggregator.

It has the following options:

Option Type Description
dataSource DataSource Mandatory: The javax.sql.DataSource to use for accessing the database.
repositoryName String Mandatory: The name of the repository.
transactionManager TransactionManager Mandatory: The org.springframework.transaction.PlatformTransactionManager to mange transactions for the database. The TransactionManager must be able to support databases.
lobHandler LobHandler A org.springframework.jdbc.support.lob.LobHandler to handle Lob types in the database. Use this option to use a vendor specific LobHandler, for example when using Oracle.
returnOldExchange boolean Whether the get operation should return the old existing Exchange if any existed. By default this option is false to optimize as we do not need the old exchange when aggregating.
useRecovery boolean Whether or not recovery is enabled. This option is by default true. When enabled the Camel Aggregator automatic recover failed aggregated exchange and have them resubmitted.
recoveryInterval long If recovery is enabled then a background task is run every x'th time to scan for failed exchanges to recover and resubmit. By default this interval is 5000 millis.
maximumRedeliveries int Allows you to limit the maximum number of redelivery attempts for a recovered exchange. If enabled then the Exchange will be moved to the dead letter channel if all redelivery attempts failed. By default this option is disabled. If this option is used then the deadLetterUri option must also be provided.
deadLetterUri String An endpoint uri for a Dead Letter Channel where exhausted recovered Exchanges will be moved. If this option is used then the maximumRedeliveries option must also be provided.

The JdbcAggregationRepository will by default recover any failed Exchange. It does this by having a background tasks that scans for failed Exchanges in the persistent store. You can use the checkInterval option to set how often this task runs. The recovery works as transactional which ensures that Camel will try to recover and redeliver the failed Exchange. Any Exchange which was found to be recovered will be restored from the persistent store and resubmitted and send out again.

The following headers is set when an Exchange is being recovered/redelivered:

Header Type Description
Exchange.REDELIVERED Boolean Is set to true to indicate the Exchange is being redelivered.
Exchange.REDELIVERY_COUNTER Integer The redelivery attempt, starting from 1.

Only when an Exchange has been successfully processed it will be marked as complete which happens when the confirm method is invoked on the AggregationRepository. This means if the same Exchange fails again it will be kept retried until it success.

You can use option maximumRedeliveries to limit the maximum number of redelivery attempts for a given recovered Exchange. You must also set the deadLetterUri option so Camel knows where to send the Exchange when the maximumRedeliveries was hit.

You can see some examples in the unit tests of camel-sql, for example this test.

Comments powered by Disqus