8.3. Topologies
8.3.1. Load-Balancing Cluster
Overview
Fabric exploits the concept of broker groups to implement cluster functionality. To set up a load-balancing cluster, all of the brokers in the cluster should register with the same group name, but using unique broker names.
For example, Figure 8.1, “Load-Balancing Cluster” shows a load-balancing cluster with the group name,
loadbal, and with three brokers registered in the group: brokerx, brokery, and brokerz. This topology is most useful in a scenario where producer is generating a heavy load but does not care if all the messages are delivered to the consumer. In this scenario non persistent messages are used. This topology does not have shared storage. For scenarios where better guarantees regarding delivery of messages is required it is best to combine networks and master slave as defined in subsequent sections.
Figure 8.1. Load-Balancing Cluster
Create brokers in a load-balancing cluster
The basic rules for creating a load-balancing cluster of brokers are as follows:
- Choose a group name for the load-balancing cluster.
- Each broker in the cluster registers with the chosen group.
- Each broker must be identified by a unique broker name.
- Normally, each broker is deployed in a separate container.
For example, consider the cluster shown in Figure 8.1, “Load-Balancing Cluster”. The group name is
loadbal and the cluster consists of three broker instances with broker names: brokerx, brokery, and brokerz.
To create this cluster, perform the following steps:
- First of all create some containers:
JBossFuse:karaf@root> container-create-child root broker 3 Creating new instance on SSH port 8102 and RMI ports 1100/44445 at: /Users/jdoe/Downloads/jboss-fuse-6.3.0.redhat-254/instances/broker2 Creating new instance on SSH port 8104 and RMI ports 1102/44447 at: /Users/jdoe/Downloads/jboss-fuse-6.3.0.redhat-254/instances/broker3 Creating new instance on SSH port 8103 and RMI ports 1101/44446 at: /Users/jdoe/Downloads/jboss-fuse-6.3.0.redhat-254/instances/broker1 The following containers have been created successfully: Container: broker2. Container: broker3. Container: broker1. - Wait until the containers are successfully provisioned. You can conveniently monitor them using the
watchcommand, as follows:JBossFuse:karaf@root> watch container-list
- You can then assign broker profiles to each of the containers, using the
fabric:mq-createcommand, as follows:JBossFuse:karaf@root> mq-create --group loadbal --assign-container broker1 brokerx MQ profile mq-broker-loadbal.brokerx ready JBossFuse:karaf@root> mq-create --group loadbal --assign-container broker2 brokery MQ profile mq-broker-loadbal.brokery ready JBossFuse:karaf@root> mq-create --group loadbal --assign-container broker3 brokerz MQ profile mq-broker-loadbal.brokerz ready
- You can use the
fabric:profile-listcommand to see the new profiles created for these brokers:JBossFuse:karaf@root> profile-list --hidden [id] [# containers] [parents] ... mq-broker-loadbal.brokerx 1 mq-base mq-broker-loadbal.brokery 1 mq-base mq-broker-loadbal.brokerz 1 mq-base mq-client-loadbal ...
- You can use the
fabric:cluster-listcommand to view the cluster configuration for this load balancing cluster:JBossFuse:karaf@root> cluster-list [cluster] [masters] [slaves] [services] ... amq/loadbal brokerx broker1 - tcp://MyLocalHost.61616 mqtt://MyLocalHost.61424 amqp://MyLocalHost.61426 stomp://MyLocalHost.61428 brokery broker2 - tcp://MyLocalHost.61437 mqtt://MyLocalHost.61439 amqp://MyLocalHost.61441 stomp://MyLocalHost.61443 brokerz broker3 - tcp://MyLocalHost.61453 mqtt://MyLocalHost.61455 amqp://MyLocalHost.61457 stomp://MyLocalHost.61459
Configure clients of a load-balancing cluster
To connect a client to a load-balancing cluster, use a URL of the form,
discovery:(fabric:GroupName), which automatically load balances the client across the available brokers in the cluster. For example, to connect a client to the loadbal cluster, you would use a URL like the following:
discovery:(fabric:loadbal)
For convenience, the
mq-create command automatically generates a profile named mq-client-GroupName, which provides an ActiveMQConnectionFactory instance in the registry. If you deploy this profile together with a Camel route that uses JMS endpoints, the Camel route will automatically find and use the ActiveMQConnectionFactory instance to connect to the broker cluster.
8.3.2. Master-Slave Cluster
Overview
In the master-slave pattern, multiple peer brokers provide the same service and all compete to be the master. Only one master can exist at a given time, while the rest remain on standby as slaves. If the master stops, the remaining brokers (slaves) compete to become the new master. If the broker containers are deployed across different machines or data centres, the result is a highly available broker.
For example, Figure 8.2, “Master-Slave Cluster” shows a master-slave cluster with the group name,
masterslave, and three brokers that compete with each other to register as the broker, hq-broker. A broker becomes the master by acquiring a lock (where the lock implementation is provided by the underlying ZooKeeper registry). The other two brokers that fail to acquire the lock remain as slaves (but they continue trying to acquire the lock, at regular time intervals).
Figure 8.2. Master-Slave Cluster
Create brokers in a master-slave cluster
The basic rules for creating a master-slave cluster of brokers are as follows:
- Choose a group name for the master-slave cluster.
- Each broker in the cluster registers with the chosen group.
- Each broker must be identified by the same virtual broker name.
- Normally, each broker is deployed in a separate container.
For example, consider the cluster shown in Figure 8.2, “Master-Slave Cluster”. The group name is
masterslave and the cluster consists of three broker instances, each with the same broker name: hq-broker. You can create this cluster by entering a single fabric:mq-create command, as follows:
JBossFuse:karaf@root> mq-create --create-container broker --replicas 3 --group masterslave hq-broker
Alternatively, if you have already created three containers,
broker1, broker2 and broker3 (possibly running on separate machines), you can deploy a cluster of three brokers to the containers by entering the following command:
JBossFuse:karaf@root> mq-create --assign-container broker1,broker2,broker3 --group masterslave hq-broker
The first broker that starts becomes the master, while the others are slaves. When you stop the master, one of the slaves will take over and clients will reconnect. If brokers are persistent, you need to ensure that they all use the same store—for details of how to configure this, see the section called “Configuring persistent data”.
Configure clients of a master-slave cluster
To connect a client to a master-slave cluster, use a URL of the form,
discovery:(fabric:GroupName), which automatically connects the client to the current master server. For example, to connect a client to the masterslave cluster, you would use a URL like the following:
discovery:(fabric:masterslave)
You can use the automatically generated client profile,
mq-client-masterslave, to create sample clients (by referencing the corresponding ActiveMQConnectionFactory instance in the registry).
Locking mechanism
One benefit of this kind of master-slave architecture is that it does not depend on shared storage for locking, so it can be used even with non-persistent brokers. The broker group uses ZooKeeper to manage a shared distributed lock that controls ownership of the master status.
Re-using containers for multiple clusters
Fabric supports re-using the same containers for multiple master-slave clusters, which is a convenient way to economize on hardware resources. For example, given the three containers,
broker1, broker2, and broker3, already running the hq-broker cluster, it is possible to reuse the same containers for another highly available broker cluster, web-broker. You can assign the web-broker profile to the existing containers with the following command:
mq-create --assign-container broker1,broker2,broker3 web-broker
This command assigns the new
web-broker profile to the same containers already running hq-broker. Fabric automatically prevents two masters from running on the same container, so the master for hq-broker will run on a different container from the master for web-broker. This arrangement makes optimal use of the available resources.
Configuring persistent data
When you run a master-slave configuration with persistent brokers, it is important to specify where your store is located, because you need to be able to access it from multiple hosts. To support this scenario, the
fabric:mq-create command enables you to specify the location of the data directory, as follows:
mq-create --assign-container broker1 --data /var/activemq/hq-broker hq-broker
The preceding command creates the
hq-broker virtual broker, which uses the /var/activemq/hq-broker directory for the data (and store) location. You can then mount some shared storage to this path and share the storage amongst the brokers in the master-slave cluster.
8.3.3. Broker Networks
Overview
It is possible to combine broker clusters with broker networks, giving you a hybrid broker network that combines the benefits of broker clusters (for example, high availability) with the benefits of broker networks (managing the flow of messages between different geographical sites).
Broker networks
A broker network in AMQ is a form of federation where brokers are linked together using network connectors. This can be used as a way of forwarding messages between different geographical locations. Messages can be forwarded either statically (where specified categories of messages are always forwarded to a specific broker), or dynamically (where messages are forwarded only in response to a client that connects to a broker and subscribes to particular queues or topics).
For more details, see "Using Networks of Brokers" from the AMQ library.
Creating network connectors
In the context of Fabric, network connectors can be created by passing the
--network option to the fabric:mq-create command.
Example broker network
Consider the scenario shown in Figure 8.3, “Broker Network with Master-Slave Clusters”.
Figure 8.3. Broker Network with Master-Slave Clusters
The figure shows two master-slave clusters:
- The first cluster has the group name,
us-west, and provides high-availability with a master-slave cluster of two brokers,us-west1andus-west2. - The second cluster has the group name,
us-east, and provides high-availability with a master-slave cluster of two brokers,us-east1andus-east2.
Network connectors link the master brokers between each of the geographical locations (there are, in fact, two network connectors in this topology: from west to east and from east to west).
To create the pair of master-slave brokers for the
us-east group (consisting of the two containers us-east1 and us-east2), you would log on to a root container running in the US East location and enter a command like the following:
mq-create --group us-east --network us-west --networks-username User --networks-password Pass --create-container us-east us-east
Where the
--network option specifies the name of the broker group you want to connect to, and the User and Pass are the credentials required to log on to the us-west broker cluster. By default, the fabric:mq-create command creates a master/slave pair of brokers.
And to create the pair of master-slave brokers for the
us-west group (consisting of the two containers us-west1 and us-west2), you would log on to a root container running in the US West location and enter a command like the following:
mq-create --group us-west --network us-east --networks-username User --networks-password Pass --create-container us-west us-west
Where
User and Pass are the credentials required to log on to the us-east broker cluster.
Note
In a real scenario, you would probably first create the containers on separate machines and then assign brokers to the containers, using the
--assign-container option in place of --create-container.
Connecting to the example broker network
At the US East location, any clients that need to connect to the broker network should use the following client URL:
discovery:(fabric:us-east)
And at the US West location, any clients that need to connect to the broker network should use the following client URL:
discovery:(fabric:us-west)
Any messages that need to be propagated between locations, from US East to US West (or from US West to US East), are transmitted over the broker network through one of the network connectors.
