3.5. Scalability and High Performance

In a scalable messaging system, brokers can support an increasing number of concurrently connected clients. In a high-performance messaging system, brokers can process messages through the system, from producers to consumers, at a high rate. In a scalable high-performance messaging system, multiple brokers can concurrently process a large volume of messages for a large number of concurrently connected clients.

You can scale up Red Hat AMQ to provide connectivity for thousands of concurrent client connections and many more destinations.

Besides the obvious components—network and system hardware, transport protocols, message compression—which you can tune to increase application performance, AMQ provides means for avoiding bottlenecks caused by large messages and for scheduling message dispatches.

As shown in Figure 3.2, “Network of Brokers Example”, the brokers in a network of brokers are connected together by network connectors, which define the broker-to-broker links that form the basis of the network. Through network connectors, a network of brokers keeps track of all active consumers, receiving notifications whenever a consumer connects to or disconnects from the network. Using the information provided through client registration, brokers can determine where and how to route messages to any consumer in a network of brokers.

The brokers use a store-and-forward delivery method to move messages between them. A broker first stores messages locally before passing them on to another broker in its network. This scheme supports the distribution of queues and topics across a network of brokers.

A network of brokers can be employed in a variety of network topologies, such as hub-and-spoke, daisy chain, or mesh.

For details, see Using Networks of Brokers on the Red Hat Customer Portal.

For details, see Master/Slave in Fault Tolerant Messaging on the Red Hat Customer Portal.

Blob(binary large object) messages provide a mechanism for robust transfers of very large files, avoiding the bottlenecks often associated with them. Retrieval of a blob message is atomic. Blob messages are transferred out-of-bounds (outside the broker application) via FTP or HTTP. The blob message does not contain the file. It is only a notification that a blob is available for retrieval. The blob message contains the producer-supplied URL of the blob's location and a helper method for acquiring an InputStream to the actual data.

Though blob transfers are more robust than stream transfers, blobs rely on an external server for data storage.

For details, see ActiveMQ in Action (Snyder, Bosanac, and Davies).

Stream messages provide a mechanism for efficiently transferring very large files, avoiding the bottlenecks often associated with them. A stream message induces a client to function as a Java IOStream. The broker chunks OutputStream data received from the producer and dispatches each chunk as a JMS message. On the consumer, a corresponding InputStream must reassemble the data chunks.

This feature employs message groups so that messages with the same JMSXGroupID value are pinned to a single consumer. Using streams with queue destinations that connect to multiple receivers, regular or exclusive, does not affect message order. However, using streams with more than one producer could interfere with message ordering.

For details, see ActiveMQ in Action (Snyder, Bosanac, and Davies).

Using properties in the org.apache.activemq.ScheduledMessage interface, you can schedule when messages are dispatched to a consumer. The broker stores scheduled messages persistently, so they survive broker failure and are dispatched upon broker startup.

Four ScheduledMessage properties appended to the transport connector enable fine-grained scheduling:

For details, see ActiveMQ in Action (Snyder, Bosanac, and Davies).

AMQ supports reliable, high-performance load balancing on queues distributed across multiple consumers. If one of the consumers fails, AMQ redelivers any unacknowledged messages to a queue's other consumers, by default, in round-robin order. When one consumer is faster than the others, it receives more messages, and when any consumer slows down, other consumers take up the slack. This enables a queue to reliably load balance processing across multiple consumer processes.

Using queues this way in a network of brokers, you can implement a grid-style processing model, in which a cluster of worker processes, in a scalable and efficient Staged Event-Driven Architecture (SEDA) way, asynchronously process messages sent to a queue.

For details, see Balancing Consumer Load in Using Networks of Brokers on the Red Hat Customer Portal.