For greater scalability on the producer side, you might want to spread the message load across multiple brokers. This can be achieved by setting up a broker network, as described in Broker Networks. For the purpose of spreading the load across brokers, one of the most useful topologies is the concentrator topology, as shown in Figure 2.22.
The concentrator topology consists of two layers of brokers, as follows:
The first layer of brokers, A, B, and C, accepts connections from message producers and specializes in receiving incoming messages;
The second layer of brokers, X and Y, accepts connections from message consumers and specializes in sending messages to the consumers.
With this topology, the first layer of brokers, A, B, and C, can focus on managing a large number of incoming producer connections. The received messages are consolidated within the brokers before being passed through a relatively small number of network connectors to the second layer, X and Y. The brokers, X and Y, only need to deal with a relatively small number of connections (assuming the number of consumers is small). If the number of consumers is large, you could add a third layer of brokers to fan out and handle the consumer connections.
When connecting to a broker network, producers and consumers must be distributed
across the available brokers. For example, in the case of a producer connecting to
the concentrator topology shown in Figure 2.22, the
producer should connect using a failover URL that shares the load across the brokers
in the first layer, A, B, and C. Assuming that the brokers are running on separate
hosts—brokerA
, brokerB
, and
brokerC
—and assuming that they all listen on IP port 61616, the
producers should use the following failover URL to connect to the broker
network:
failover://(tcp://brokerA:61616,tcp://brokerB:61616,tcp://brokerC:61616)
By default, a producer will randomly select one of the URLs in the failover list and attempt to connect to it. If the first connection attempt fails, the producer will try the other URLs in turn. For more details of the failover protocol, see Failover Protocol.