Chapter 7. Configuring maximum memory usage for addresses
AMQ Broker transparently supports huge queues containing millions of messages, even if the machine that is hosting the broker is running with limited memory.
In these situations, it might be not possible to store all of the queues in memory at any one time. To protect against excess memory consumption, you can configure the maximum memory usage that is allowed for each address on the broker. In addition, you can specify what action the broker takes when this limit is reached for a given address.
In particular, when memory usage for an address reaches the configured limit, you can configure the broker to take one of the following actions:
- Page messages
- Silently drop messages
- Drop messages and notify the sending clients
- Block clients from sending messages
The sections that follow show how to configure maximum memory usage for addresses and the corresponding actions that the broker can take when the limit for an address is reached.
When you use transactions, the broker might allocate extra memory to ensure transactional consistency. In this case, the memory usage reported by the broker might not reflect the total number of bytes being used in memory. Therefore, if you configure the broker to page, drop, or block messages based on a specified maximum memory usage, you should not also use transactions.
7.1. Configuring message paging
For any address that has a maximum memory usage limit specified, you can also specify what action the broker takes when that usage limit is reached. One of the options that you can configure is paging.
If you configure the paging option, when the maximum size of an address is reached, the broker starts to store messages for that address on disk, in files known as page files. Each page file has a maximum size that you can configure. Each address that you configure in this way has a dedicated folder in your file system to store paged messages.
Both queue browsers and consumers can navigate through page files when inspecting messages in a queue. However, a consumer that is using a very specific filter might not be able to consume a message that is stored in a page file until existing messages in the queue have been consumed first. For example, suppose that a consumer filter includes a string expression such as "color='red'"
. If a message that meets this condition follows one million messages with the property "color='blue'"
, the consumer cannot consume the message until those with "color='blue'"
have been consumed first.
The broker transfers (that is, depages) messages from disk into memory when clients are ready to consume them. The broker removes a page file from disk when all messages in that file have been acknowledged.
The procedures that follow show how to configure message paging.
7.1.1. Specifying a paging directory
The following procedure shows how to specify the location of the paging directory.
Procedure
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. Within the
core
element, add thepaging-directory
element. Specify a location for the paging directory in your file system.<configuration ...> <core ...> ... <paging-directory>/path/to/paging-directory</paging-directory> ... </core> </configuration>
For each address that you subsequently configure for paging, the broker adds a dedicated directory within the paging directory that you have specified.
7.1.2. Configuring an address for paging
The following procedure shows how to configure an address for paging.
Prerequisites
- You should be familiar with how to configure addresses and address settings. For more information, see Chapter 4, Configuring addresses and queues.
Procedure
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. For an
address-setting
element that you have configured for a matching address or set of addresses, add configuration elements to specify maximum memory usage and define paging behavior. For example:<address-settings> <address-setting match="my.paged.address"> ... <max-size-bytes>104857600</max-size-bytes> <page-size-bytes>10485760</page-size-bytes> <address-full-policy>PAGE</address-full-policy> ... </address-setting> </address-settings>
max-size-bytes
-
Maximum size, in bytes, of the memory allowed for the address before the broker executes the policy specified for
address-full-policy
. The default value is-1
, which means that there is no limit. The value that you specify also supports byte notation such as "K", "MB", and "GB". page-size-bytes
-
Size, in bytes, of each page file used on the paging system. The default value is
10485760
(that is, 10 MiB). The value that you specify also supports byte notation such as "K", "MB", and "GB". address-full-policy
Action that the broker takes when then the maximum size for an address has been reached. The default value is
PAGE
. Valid values are:PAGE
- The broker pages any further messages to disk.
DROP
- The broker silently drops any further messages.
FAIL
- The broker drops any further messages and issues exceptions to client message producers.
BLOCK
- Client message producers block when they try to send further messages.
Additional paging configuration elements that are not shown in the preceding example are described below.
page-max-cache-size
-
Number of page files that the broker keeps in memory to optimize IO during paging navigation. The default value is
5
. page-sync-timeout
-
Time, in nanoseconds, between periodic page synchronizations. If you are using an asynchronous IO journal (that is,
journal-type
is set toASYNCIO
in thebroker.xml
configuration file), the default value is3333333
. If you are using a standard Java NIO journal (that is,journal-type
is set toNIO
), the default value is the configured value of thejournal-buffer-timeout
parameter.
In the preceding example , when messages sent to the address my.paged.address
exceed 104857600 bytes in memory, the broker begins paging.
If you specify max-size-bytes
in an address-setting
element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value of max-size-bytes
.
7.1.3. Configuring a global paging size
Sometimes, configuring a memory limit per address is not practical, for example, when a broker manages many addresses that have different usage patterns. In these situations, you can specify a global memory limit. The global limit is the total amount of memory that the broker can use for all addresses. When this memory limit is reached, the broker executes the policy specified for address-full-policy
for the address associated with a new incoming message.
The following procedure shows how to configure a global paging size.
Prerequisites
- You should be familiar with how to configure an address for paging. For more information, see Section 7.1.2, “Configuring an address for paging”.
Procedure
Stop the broker.
On Linux:
<broker_instance_dir>/bin/artemis stop
On Windows:
<broker_instance_dir>\bin\artemis-service.exe stop
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. Within the
core
element, add theglobal-max-size
element and specify a value. For example:<configuration> <core> ... <global-max-size>1GB</global-max-size> ... </core> </configuration>
global-max-size
Total amount of memory, in bytes, that the broker can use for all addresses. When this limit is reached, for the address associated with an incoming message, the broker executes the policy that is specified as a value for
address-full-policy
. The default value ofglobal-max-size
is half of the maximum memory available to the Java virtual machine (JVM) that is hosting the broker.The value for
global-max-size
is in bytes, but also supports byte notation (for example, "K", "Mb", "GB").In the preceding example, the broker is configured to use a maximum of one gigabyte of available memory when processing messages.
Start the broker.
On Linux:
<broker_instance_dir>/bin/artemis run
On Windows:
<broker_instance_dir>\bin\artemis-service.exe start
7.1.4. Limiting disk usage during paging
You can limit the amount of physical disk space that the broker can use before it blocks incoming messages rather than paging them.
The following procedure shows how to set a limit for disk usage during paging.
Procedure
Stop the broker.
On Linux:
<broker_instance_dir>/bin/artemis stop
On Windows:
<broker_instance_dir>\bin\artemis-service.exe stop
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. Within the
core
element add themax-disk-usage
configuration element and specify a value. For example:<configuration> <core> ... <max-disk-usage>50</max-disk-usage> ... </core> </configuration>
max-disk-usage
Maximum percentage of the available disk space that the broker can use when paging messages. When this limit is reached, the broker blocks incoming messages rather than paging them. The default value is
90
.In the preceding example, the broker is limited to using fifty percent of disk space when paging messages.
Start the broker.
On Linux:
<broker_instance_dir>/bin/artemis run
On Windows:
<broker_instance_dir>\bin\artemis-service.exe start
7.2. Configuring message dropping
Section 7.1.2, “Configuring an address for paging” shows how to configure an address for paging. As part of that procedure, you set the value of address-full-policy
to PAGE
.
To drop messages (rather than paging them) when an address reaches its specified maximum size, set the value of the address-full-policy
to one of the following:
DROP
- When the maximum size of a given address has been reached, the broker silently drops any further messages.
FAIL
- When the maximum size of a given address has been reached, the broker drops any further messages and issues exceptions to producers.
7.3. Configuring message blocking
The following procedures show how to configure message blocking when a given address reaches the maximum size limit that you have specified.
You can configure message blocking only for the Core, OpenWire, and AMQP protocols.
7.3.1. Blocking Core and OpenWire producers
The following procedure shows how to configure message blocking for Core and OpenWire message producers when a given address reaches the maximum size limit that you have specified.
Prerequisites
- You should be familiar with how to configure addresses and address settings. For more information, see Chapter 4, Configuring addresses and queues.
Procedure
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. For an
address-setting
element that you have configured for a matching address or set of addresses, add configuration elements to define message blocking behavior. For example:<address-settings> <address-setting match="my.blocking.address"> ... <max-size-bytes>300000</max-size-bytes> <address-full-policy>BLOCK</address-full-policy> ... </address-setting> </address-settings>
max-size-bytes
Maximum size, in bytes, of the memory allowed for the address before the broker executes the policy specified for
address-full-policy
. The value that you specify also supports byte notation such as "K", "MB", and "GB".NoteIf you specify
max-size-bytes
in anaddress-setting
element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value ofmax-size-bytes
.address-full-policy
- Action that the broker takes when then the maximum size for an address has been reached.
In the preceding example, when messages sent to the address my.blocking.address
exceed 300000 bytes in memory, the broker begins blocking further messages from Core or OpenWire message producers.
7.3.2. Blocking AMQP producers
Protocols such as Core and OpenWire use a window-size flow control system. In this system, credits represent bytes and are allocated to producers. If a producer wants to send a message, the producer must wait until it has sufficient credits for the size of the message.
By contrast, AMQP flow control credits do not represent bytes. Instead, AMQP credits represent the number of messages a producer is permitted to send, regardless of message size. Therefore, it is possible, in some situations, for AMQP producers to significantly exceed the max-size-bytes
value of an address.
Therefore, to block AMQP producers, you must use a different configuration element, max-size-bytes-reject-threshold
. For a matching address or set of addresses, this element specifies the maximum size, in bytes, of all AMQP messages in memory. When the total size of all messages in memory reaches the specified limit, the broker blocks AMQP producers from sending further messages.
The following procedure shows how to configure message blocking for AMQP message producers.
Prerequisites
- You should be familiar with how to configure addresses and address settings. For more information, see Chapter 4, Configuring addresses and queues.
Procedure
-
Open the
<broker_instance_dir>/etc/broker.xml
configuration file. For an
address-setting
element that you have configured for a matching address or set of addresses, specify the maximum size of all AMQP messages in memory. For example:<address-settings> <address-setting match="my.amqp.blocking.address"> ... <max-size-bytes-reject-threshold>300000</max-size-bytes-reject-threshold> ... </address-setting> </address-settings>
max-size-bytes-reject-threshold
Maximum size, in bytes, of the memory allowed for the address before the broker blocks further AMQP messages. The value that you specify also supports byte notation such as "K", "MB", and "GB". By default,
max-size-bytes-reject-threshold
is set to-1
, which means that there is no maximum size.NoteIf you specify
max-size-bytes-reject-threshold
in anaddress-setting
element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value ofmax-size-bytes-reject-threshold
.
In the preceding example, when messages sent to the address my.amqp.blocking.address
exceed 300000 bytes in memory, the broker begins blocking further messages from AMQP producers.
7.4. Understanding memory usage on multicast addresses
When a message is routed to an address that has multicast queues bound to it, there is only one copy of the message in memory. Each queue has only a reference to the message. Because of this, the associated memory is released only after all queues referencing the message have delivered it.
In this type of situation, if you have a slow consumer, the entire address might experience a negative performance impact.
For example, consider this scenario:
- An address has ten queues that use the multicast routing type.
- Due to a slow consumer, one of the queues does not deliver its messages. The other nine queues continue to deliver messages and are empty.
- Messages continue to arrive to the address. The queue with the slow consumer continues to accumulate references to the messages, causing the broker to keep the messages in memory.
- When the maximum size of the address is reached, the broker starts to page messages.
In this scenario because of a single slow consumer, consumers on all queues are forced to consume messages from the page system, requiring additional IO.
Additional resources
- To learn how to configure flow control to regulate the flow of data between the broker and producers and consumers, see Flow control in the AMQ Core Protocol JMS documentation.