Fuse ESB - Clustering Guide - Balancing Consumer Load

Overview

Depending on the topology of your broker network, the default conduit subscription behavior can sometimes be detrimental to load balancing on the consumer side. This section describes the nature of the problem and explains how to alter the network configuration to achieve optimum load balancing.

Default load behavior

When the conduit subscriptions option is enabled (which it is by default), this can result in an uneven distribution of messages between consumers that subscribe to a particular queue. Figure 2.20 illustrates how this uneven distribution can come about.

Figure 2.20. Message Flow when Conduit Subscriptions Enabled

Assume that the consumers, C1, C2, and C3, all subscribe to the same queue, TEST.FOO. Now consider what happens when producer, P, connects to broker A and sends 12 messages to the queue, TEST.FOO. Because conduit subscriptions is enabled, broker A sees only a single (conduit) subscription from broker B. Broker A also sees a single subscription from consumer C1. So, broker A will send messages alternately to C1 and B, sending a total of 6 messages to C1 and 6 messages to B. Now broker B sees two subscriptions, from C2 and C3 respectively. So, broker B will send messages alternately to C2 and C3, sending a total of 3 message to C2 and 3 messages to C3.

In the end, the distribution of messages amongst the consumers is 6, 3, 3, which is not optimally load balanced.

Disabling conduit subscriptions

If you want to improve the load balancing behavior for queues, you can disable conduit subscriptions by setting conduitSubscriptions to false. For example, you can disable conduit subscriptions on a broker connector as follows:

<networkConnectors>
    <networkConnector name="linkToBrokerB"
       uri="static:(tcp://localhost:61002)"
       networkTTL="3"
       conduitSubscriptions="false" />
</networkConnectors>

	Warning
	Be careful, if you are using topics. Disabling conduit subscriptions can lead to duplicate topic messages—see Conduit subscriptions.

Balanced load behavior

When the conduit subscriptions option is disabled, you can achieve optimal distribution of messages between consumers that subscribe to a particular queue. Figure 2.21 illustrates the message flow in this case.

Figure 2.21. Message Flow when Conduit Subscriptions Disabled

Assume that the consumers, C1, C2, and C3, all subscribe to the same queue, TEST.FOO. Now consider what happens when producer, P, connects to broker A and sends 12 messages to the queue, TEST.FOO. Because conduit subscriptions is disabled, broker A sees two subscriptions on broker B and a single subscription from consumer C1. So, by alternating between all the subscriptions, broker A ends up sending 4 messages to C1 and 8 messages to broker B. Broker B then sends 4 messages each to consumers C2 and C3.

In the end, the distribution of messages amongst the consumers is 4, 4, 4, which is optimally balanced.

Separate connectors for topics and queues

In some cases, you might need to disable conduit subscriptions for queues (in order to optimize load balancing), but also enable conduit subscriptions for topics (to avoid duplicate topic messages). You cannot configure this using a single network connector, because the conduitSubscriptions flag applies simultaneously to queues and topics. On the other hand, it is possible to configure topics and queues differently, if you create multiple network connectors: one for queues and another for topics (see Figure 2.4).

Example 2.5 shows how to configure separate network connectors for topics and queues. The queuesOnly network connector, which has conduit subscriptions enabled, is equipped with a filter that transmits only queue messages. The topicsOnly network connector, which has conduit subscriptions disabled, is equipped with a filter that transmits only topic messages.

Example 2.5. Separate Configuration of Topics and Queues

<networkConnectors>
    <networkConnector name="queuesOnly"
       uri="static:(tcp://localhost:61002)"
       networkTTL="3"
       conduitSubscriptions="false" >
        <dynamicallyIncludedDestinations>
            <queue physicalName=">"/>
        </dynamicallyIncludedDestinations>
    </networkConnector>

    <networkConnector name="topicsOnly"
       uri="static:(tcp://localhost:61002)"
       networkTTL="3"
       conduitSubscriptions="true" >
        <dynamicallyIncludedDestinations>
            <topic physicalName=">"/>
        </dynamicallyIncludedDestinations>
    </networkConnector>
</networkConnectors>