Chapter 9. Load Balancing
Abstract
Broker networks can address the problem of load balancing in a messaging system. Consumer load is managed by changing how network connectors recognize subscriptions. Producer load is managed using different broker topologies.
9.1. Balancing Consumer Load
Abstract
When using queues it is easy to balance load over a group of consumers. The messages are evenly distributed among all of the consumers attached to a queue. In a network of brokers, however, conduit subscriptions can adversely effect the ability of brokers to evenly distribute messages to all of the queue subscribers. This can be mitigated by disabling conduit subscriptions.
Overview
Multiple consumers attached to a JMS queue automatically obey competing consumer semantics. That is, each message transmitted by the queue is consumed by one consumer only. Hence, if you want to scale up load balancing on the consumer side, all that you need to do is attach extra consumers to the queue. The competing consumer semantics of the JMS queue then automatically ensures that the queue's messages are evenly distributed amongst the attached consumers.
The default behavior of Red Hat JBoss A-MQ's conduit subscriptions, however, can sometimes be detrimental to load balancing on the consumer side. As described in the section called “Conduit subscriptions”, conduit subscriptions concentrate all of the subscriptions from a networked broker into a single subscription. For topics this behavior optimizes traffic and has no effect on consumer load. For queues, however, it results in uneven message distribution which can impede consumer load balancing.
Default load behavior
Figure 9.1, “Message Flow when Conduit Subscriptions Enabled” illustrates how conduit subscriptions can result in uneven message distribution to the consumers of a queue.
Figure 9.1. Message Flow when Conduit Subscriptions Enabled
Assume that the consumers, C1, C2, and C3, all subscribe to the
TEST.FOO queue. Producer, P, connects to Broker A and sends 12 messages to the TEST.FOO queue. By default conduit subscriptions are enabled and Broker A sees only a single subscription from Broker B and a single subscription from consumer C1. So, Broker A sends messages alternately to C1 and B. Assuming that C1 and B process messages at the same speed, A sends a total of 6 messages to C1 and 6 messages to B.
Broker B sees two subscriptions, from C2 and C3 respectively. So, Broker B will send messages alternately to C2 and C3. Assuming that both consumers process messages at equal speed, each consumer receives a total of 3 messages.
In the end, the distribution of messages amongst the consumers is 6, 3, 3, which is not optimally load balanced. C1 processes twice as many messages as either C2 or C3.
Disabling conduit subscriptions
If you want to improve the load balancing behavior for queues, you can disable conduit subscriptions by setting the
networkConnector element's conduitSubscriptions to false. Example 9.1, “Disabling Conduit Subscriptions” shows configuration for a network connector with conduit subscriptions disabled.
Example 9.1. Disabling Conduit Subscriptions
<networkConnectors>
<networkConnector name="linkToBrokerB"
uri="static:(tcp://localhost:61002)"
networkTTL="3"
conduitSubscriptions="false" />
</networkConnectors>Warning
As described in the section called “Conduit subscriptions”, conduit subscriptions protect against duplicate topic messages. If you are using both queues and topics consider using separate network connectors for queues and topics. See the section called “Separate connectors for topics and queues”.
Balanced load behavior
Figure 9.2, “Message Flow when Conduit Subscriptions Disabled” illustrates the message flow through a queue with distributed consumers when conduit subscriptions are disabled.
Figure 9.2. Message Flow when Conduit Subscriptions Disabled
Assume that the consumers, C1, C2, and C3, all subscribe to the
TEST.FOO queue. Producer, P, connects to Broker A and sends 12 messages to the TEST.FOO queue. With conduit subscriptions disabled, Broker A sees both of the subscriptions on Broker B and a single subscription from consumer C1. Broker A sends messages alternately to each of the subscriptions. Assuming that all of the consumers process messages at equal speeds, C1 receives 4 messages and Broker B receives 8 messages.
Broker B sees two subscriptions, from C2 and C3 respectively. So, Broker B will send messages alternately to C2 and C3. Assuming that both consumers process messages at equal speed, each consumer receives a total of 4 messages.
In the end, the distribution of messages amongst the consumers is 4, 4, 4, which is optimally balanced.
Separate connectors for topics and queues
If your brokers need to handle both queues and topics, you might need to disable conduit subscriptions for queues to optimize load balancing, but also enable conduit subscriptions for topics to avoid duplicate topic messages.
Because the
conduitSubscriptions attribute applies simultaneously to queues and topics, you cannot configure this using a single network connector. It is possible to configure topics and queues differently by using multiple network connectors: one for queues and another for topics.
Example 9.2, “Separate Configuration of Topics and Queues” shows how to configure separate network connectors for topics and queues. The
queuesOnly network connector, which has conduit subscriptions disabled, is equipped with a filter that transmits only queue messages. The topicsOnly network connector, which has conduit subscriptions enabled, is equipped with a filter that transmits only topic messages.
Example 9.2. Separate Configuration of Topics and Queues
<networkConnectors>
<networkConnector name="queuesOnly"
uri="static:(tcp://localhost:61002)"
networkTTL="3"
conduitSubscriptions="false">
<dynamicallyIncludedDestinations>
<queue physicalName=">"/>
</dynamicallyIncludedDestinations>
</networkConnector>
<networkConnector name="topicsOnly"
uri="static:(tcp://localhost:61002)"
networkTTL="3">
<dynamicallyIncludedDestinations>
<topic physicalName=">"/>
</dynamicallyIncludedDestinations>
</networkConnector>
</networkConnectors>