Search

2.7. Horizontal Scaling

download PDF

Overview

Horizontal scaling refers to the strategy of increasing capacity by adding multiple brokers to your messaging network. Figure 2.2, “Scaling with Multiple Brokers” illustrates how a broker network can be used to support a large number of consumers.

Figure 2.2. Scaling with Multiple Brokers

Scaling with Multiple Brokers

Broker networks

You can improve the scalability of your messaging system by adding multiple brokers to the system, thus escaping the inherent resource limits of a broker deployed on a single machine. Brokers can be combined into a network by adding network connectors between the brokers, which enables you to define broker networks with an arbitrary topology.
When brokers are linked together as a network, routes from producers to consumers are created dynamically, as clients connect to and disconnect from the network. That is, with the appropriate topology, a consumer can connect to any broker in the network and the network automatically routes messages from producers attached at any other point in the network.
For a detailed discussion of how to set up broker networks, see "Using Networks of Brokers".

Static scales better than dynamic

Red Hat JBoss A-MQ offers two alternative strategies for routing messages through a broker network: static propagation and dynamic propagation.
Although dynamic propagation is more flexible, it necessitates sending advisory messages throughout the broker network, which the brokers then use to figure out the optimal route in a dynamic way. As you scale up the network, there is a danger that the advisory messages could swamp the traffic in the broker network.
Static propagation requires you to specify routes explicitly, by telling the broker where to forward messages for specific queues and topics (you can use pattern matching). In this case, you can configure the brokers to disable advisory messages altogether, which eliminates the scalability problems associated with advisory messages.

Asynchronous network connection establishment

By default, a broker establishes connections to its peers in the network using a single thread. If the broker connects to a large number of peers, however, the single-threaded approach can result in a very slow broker start-up time. For example, if one or more of the peers responds slowly, the initiating broker has to wait until that slow connection is established before proceeding to establish the remaining connections.
In this case, you can accelerate the broker start-up by enabling asynchronous network connection establishment. This feature employs a thread pool to establish network connections in parallel. Set the networkConnectorStartAsync attribute on the broker element to true, as follows:
<beans ...>
  <broker ... networkConnectorStartAsync="true">...</broker>
</beans>

Client-side traffic partitioning

An alternative horizontal scaling strategy is to deploy multiple brokers, but to leave them isolated from one another, so that there is no broker network. In this case, you can leave it up to the clients to decide which broker to send messages to or receive messages from. This requires messages to be partitioned into different categories (for example, based on the destination name), so that specific message categories are sent to a particular broker.
The advantages of this approach are:
  • You can use all the tuning techniques for vertical scaling.
  • You can achieve better horizontal scalability than a network of brokers (because there is less broker crosstalk).
The disadvantage of this approach is that your clients are slightly more complex, because they must implement the partitioning of messages into different categories and the selection of the appropriate broker.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.