Search
SupportConsoleDevelopersStart a trialContact

Chapter 7. Configuring memory usage for addresses

download PDF

AMQ Broker transparently supports huge queues containing millions of messages, even if the machine that is hosting the broker is running with limited memory.

In these situations, it might be not possible to store all of the queues in memory at any one time. To protect against excess memory consumption, you can configure the maximum memory usage that is allowed for each address on the broker. In addition, you can configure the broker to take one of the following actions when memory usage for an address reaches the configured limit:

  • Page messages
  • Silently drop messages
  • Drop messages and notify the sending clients
  • Block clients from sending messages

If you configure the broker to page messages when the maximum memory usage for an address is reached, you can configure limits for specific addresses to:

  • Limit the disk space used to page incoming messages
  • Limit the memory used for paged messages that the broker transfers from disk back to memory when clients are ready to consume messages.

You can also set a disk usage threshold, which overrides all the configured paging limits. If the disk usage threshold is reached, the broker stops paging and blocks all incoming messages.

Important

When you use transactions, the broker might allocate extra memory to ensure transactional consistency. In this case, the memory usage reported by the broker might not reflect the total number of bytes being used in memory. Therefore, if you configure the broker to page, drop, or block messages based on a specified maximum memory usage, you should not also use transactions.

7.1. Configuring message paging

For any address that has a maximum memory usage limit specified, you can also specify what action the broker takes when that usage limit is reached. One of the options that you can configure is paging.

If you configure the paging option, when the maximum size of an address is reached, the broker starts to store messages for that address on disk, in files known as page files. Each page file has a maximum size that you can configure. Each address that you configure in this way has a dedicated folder in your file system to store paged messages.

Both queue browsers and consumers can navigate through page files when inspecting messages in a queue. However, a consumer that is using a very specific filter might not be able to consume a message that is stored in a page file until existing messages in the queue have been consumed first. For example, suppose that a consumer filter includes a string expression such as "color='red'". If a message that meets this condition follows one million messages with the property "color='blue'", the consumer cannot consume the message until those with "color='blue'" have been consumed first.

The broker transfers (that is, depages) messages from disk into memory when clients are ready to consume them. The broker removes a page file from disk when all messages in that file have been acknowledged.

The procedures that follow show how to configure message paging.

7.1.1. Specifying a paging directory

The following procedure shows how to specify the location of the paging directory.

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.

  2. Within the core element, add the paging-directory element. Specify a location for the paging directory in your file system. 

    <configuration ...>
  <core ...>
    ...
    <paging-directory>/path/to/paging-directory</paging-directory>
    ...
  </core>
</configuration>

    For each address that you subsequently configure for paging, the broker adds a dedicated directory within the paging directory that you have specified.

7.1.2. Configuring an address for paging

The following procedure shows how to configure an address for paging.

Prerequisites

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.

  2. For an address-setting element that you have configured for a matching address or set of addresses, add configuration elements to specify maximum memory usage and define paging behavior. For example: 

    <address-settings>
    <address-setting match="my.paged.address">
        ...
        <max-size-bytes>104857600</max-size-bytes>
        <max-size-messages>20000</max-size-messages>
        <page-size-bytes>10485760</page-size-bytes>
        <address-full-policy>PAGE</address-full-policy>
        ...
    </address-setting>
</address-settings>
    max-size-bytes
    Maximum size, in bytes, of the memory allowed for the address before the broker executes the action specified for the address-full-policy attribute. The default value is -1, which means that there is no limit. The value that you specify also supports byte notation such as "K", "MB", and "GB".
    max-size-messages
    Maximum number of messages allowed for the address before the broker executes the action specified for the address-full-policy attribute. The default value is -1, which means that there is no message limit.
    page-size-bytes
    Size, in bytes, of each page file used on the paging system. The default value is 10485760 (that is, 10 MiB). The value that you specify also supports byte notation such as "K", "MB", and "GB".
    address-full-policy

    Action that the broker takes when the maximum size for an address has been reached. The default value is PAGE. Valid values are:

    PAGE
    The broker pages any further messages to disk.
    DROP
    The broker silently drops any further messages.
    FAIL
    The broker drops any further messages and issues exceptions to client message producers.
    BLOCK
    Client message producers block when they try to send further messages.

    If you set limits for the max-size-bytes and max-size-message attributes, the broker executes the action specified for the address-full-policy attribute when either limit is reached. With the configuration in the previous example, the broker starts paging messages for the my.paged.address address when the total messages for the address in memory exceeds 20,000 or uses 104857600 bytes of available memory.

    Additional paging configuration elements that are not shown in the preceding example are described below.

    page-sync-timeout
    Time, in nanoseconds, between periodic page synchronizations. If you are using an asynchronous IO journal (that is, journal-type is set to ASYNCIO in the broker.xml configuration file), the default value is 3333333. If you are using a standard Java NIO journal (that is, journal-type is set to NIO), the default value is the configured value of the journal-buffer-timeout parameter.

In the preceding example , when messages sent to the address my.paged.address exceed 104857600 bytes in memory, the broker begins paging.

Note

If you specify max-size-bytes in an address-setting element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value of max-size-bytes.

7.1.3. Configuring a global paging size

Sometimes, configuring a memory limit per address is not practical, for example, when a broker manages many addresses that have different usage patterns. In these situations, you can specify a global memory limit. The global limit is the total amount of memory that the broker can use for all addresses. When this memory limit is reached, the broker executes the action specified for the address-full-policy attribute for the address associated with each new incoming message.

The following procedure shows how to configure a global paging size.

Prerequisites

Procedure

  1. Stop the broker.

    1. On Linux: 

      <broker_instance_dir>/bin/artemis stop

    2. On Windows: 

      <broker_instance_dir>\bin\artemis-service.exe stop
  2. Open the <broker_instance_dir>/etc/broker.xml configuration file.

  3. Within the core element, add the global-max-size element and specify a value. For example: 

    <configuration>
  <core>
    ...
    <global-max-size>1GB</global-max-size>
    <global-max-messages>900000</global-max-messages>
    ...
  </core>
</configuration>
    global-max-size

    Total amount of memory, in bytes, that the broker can use for all addresses. When this limit is reached, the broker executes the action specified for the address-full-policy attribute for the address associated with each incoming message. The default value of global-max-size is half of the maximum memory available to the Java virtual machine (JVM) that is hosting the broker.

    The value for global-max-size is in bytes, but also supports byte notation (for example, "K", "Mb", "GB").

    In the preceding example, the broker is configured to use a maximum of one gigabyte of available memory when processing messages.

    global-max-messages

    The total number of messages allowed for all addresses. When this limit is reached, the broker executes the action specified for the address-full-policy attribute for the address associated with each incoming message. The default value is -1, which means that there is no message limit.

    If you set limits for the global-max-size and global-max-messages attributes, the broker executes the action specified for the address-full-policy attribute when either limit is reached. With the configuration in the previous example, the broker starts paging messages for all addresses when the number of messages in memory exceeds 900,000 or uses 1 GB of available memory.

    Note

    If limits that are set for an individual address, by using the max-size-bytes or max-size-message attributes, are reached before the limits set for the global-max-size or global-max-messages attributes, the broker executes the action specified for the address-full-policy attribute for that address.

  4. Start the broker.

    1. On Linux: 

      <broker_instance_dir>/bin/artemis run

    2. On Windows: 

      <broker_instance_dir>\bin\artemis-service.exe start

7.1.4. Limiting disk usage during paging for specific addresses

You can limit the amount of disk space that the broker can use before it stops paging incoming messages for an individual address or set of addresses.

Procedure

  1. Stop the broker.

    1. On Linux: 

      <broker_instance_dir>/bin/artemis stop

    2. On Windows: 

      <broker_instance_dir>\bin\artemis-service.exe stop
  2. Open the <broker_instance_dir>/etc/broker.xml configuration file.

  3. Within the core element, add attributes to specify paging limits based on disk usage or number of messages, or both, and specify the action to take if either limit is reached. For example: 

    <address-settings>
   <address-setting match="match="my.paged.address"">
    ...
    <page-limit-bytes>10G</page-limit-bytes>
    <page-limit-messages>1000000</page-limit-messages>
    <page-full-policy>FAIL</page-full-policy>
    ...
   </address-setting>
</address-settings>
    page-limit-bytes
    Maximum size, in bytes, of the disk space allowed for paging incoming messages for the address before the broker executes the action specified for the page-full-policy attribute. The value that you specify supports byte notation such as "K", "MB", and "GB". The default value is -1, which means that there is no limit.
    page-limit-messages
    Maximum number of incoming message that can be paged for the address before the broker executes the action specified for the page-full-policy attribute. The default value is -1, which means that there is no message limit.
    page-full-policy

    Action that the broker takes when a limit set in the page-limit-bytes or page-limit-messages attributes is reached for an address. Valid values are:

    DROP The broker silently drops any further messages.

    FAIL The broker drops any further messages and notifies the sending clients

    In the preceding example, the broker pages message for the my.paged.address address until paging uses 10GB of disk space or until a total of one million messages are paged.

  4. Start the broker.

    1. On Linux: 

      <broker_instance_dir>/bin/artemis run

    2. On Windows: 

      <broker_instance_dir>\bin\artemis-service.exe start

7.1.5. Controlling the flow of paged messages into memory

If AMQ Broker is configured to page messages to disk, the broker reads paged messages and transfers the messages into memory when clients are ready to consume messages. To prevent messages from consuming excess memory, you can limit the memory used by each address for messages that the broker transfers from disk to memory.

Important

If client applications leave too many messages pending acknowledgment, the broker does not read paged messages until the pending messages are acknowledged, which can cause message starvation on the broker.

For example, if the limit for the transfer of paged messages into memory, which is 20MB by default, is reached, the broker waits for an acknowledgment from a client before it reads any more messages. If, at the same time, clients are waiting to receive sufficient messages before they send an acknowledgment to the broker, which is determined by the batch size used by clients, the broker is starved of messages.

To avoid starvation, either increase the broker limits that control the transfer of paged message into memory or reduce the number of delivering messages. You can reduce the number of delivering messages by ensuring that clients either commit message acknowledgments sooner or use a timeout and commit acknowledgments when no more messages are received from the broker.

You can see the number and size of delivering messages in a queue’s Delivering Count and Delivering Bytes metrics in AMQ Management Console.

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.
  2. For an address-settings element that you have configured for a matching address or set of addresses, specify limits on the transfer of paged messages into memory. For example: 
address-settings>
    <address-setting match="my.paged.address">
        ...
        <max-read-page-messages>104857600</max-read-page-messages>
        <max-read-page-bytes>20MB</max-read-page-bytes>
        ...
    </address-setting>
</address-settings>

Max-read-page-messages Maximum number of paged messages that the broker can read from disk into memory per-address. The default value is -1, which means that no limit applies.

Max-read-page-bytes Maximum size in bytes of paged messages that the broker can read from disk into memory per-address. The default value is 20MB.

Note

If you specify limits for both the max-read-page-messages and max-read-page-bytes attributes, the broker stops reading messages when either limit is reached.

7.1.6. Setting a disk usage threshold

You can set a disk usage threshold which, if reached, causes the broker to stop paging and block all incoming messages.

Procedure

  1. Stop the broker.

    1. On Linux: 

      <broker_instance_dir>/bin/artemis stop

    2. On Windows: 

      <broker_instance_dir>\bin\artemis-service.exe stop
  2. Open the <broker_instance_dir>/etc/broker.xml configuration file.

  3. Within the core element add the max-disk-usage configuration element and specify a value. For example: 

    <configuration>
  <core>
    ...
    <max-disk-usage>80</max-disk-usage>
    ...
  </core>
</configuration>
    max-disk-usage

    Maximum percentage of the available disk space that the broker can use. When this limit is reached, the broker blocks incoming messages. The default value is 90.

    In the preceding example, the broker is limited to using eighty percent of available disk space.

  4. Start the broker.

    1. On Linux: 

      <broker_instance_dir>/bin/artemis run

    2. On Windows: 

      <broker_instance_dir>\bin\artemis-service.exe start

7.2. Configuring message dropping

Section 7.1.2, “Configuring an address for paging” shows how to configure an address for paging. As part of that procedure, you set the value of address-full-policy to PAGE.

To drop messages (rather than paging them) when an address reaches its specified maximum size, set the value of the address-full-policy to one of the following:

DROP
When the maximum size of a given address has been reached, the broker silently drops any further messages.
FAIL
When the maximum size of a given address has been reached, the broker drops any further messages and issues exceptions to producers.

7.3. Configuring message blocking

The following procedures show how to configure message blocking when a given address reaches the maximum size limit that you have specified.

Note

You can configure message blocking only for the Core, OpenWire, and AMQP protocols.

7.3.1. Blocking Core and OpenWire producers

The following procedure shows how to configure message blocking for Core and OpenWire message producers when a given address reaches the maximum size limit that you have specified.

Prerequisites

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.

  2. For an address-setting element that you have configured for a matching address or set of addresses, add configuration elements to define message blocking behavior. For example: 

    <address-settings>
    <address-setting match="my.blocking.address">
        ...
        <max-size-bytes>300000</max-size-bytes>
        <address-full-policy>BLOCK</address-full-policy>
        ...
    </address-setting>
</address-settings>
    max-size-bytes

    Maximum size, in bytes, of the memory allowed for the address before the broker executes the policy specified for address-full-policy. The value that you specify also supports byte notation such as "K", "MB", and "GB".

    Note

    If you specify max-size-bytes in an address-setting element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value of max-size-bytes.

    address-full-policy
    Action that the broker takes when then the maximum size for an address has been reached.

In the preceding example, when messages sent to the address my.blocking.address exceed 300000 bytes in memory, the broker begins blocking further messages from Core or OpenWire message producers.

7.3.2. Blocking AMQP producers

Protocols such as Core and OpenWire use a window-size flow control system. In this system, credits represent bytes and are allocated to producers. If a producer wants to send a message, the producer must wait until it has sufficient credits for the size of the message.

By contrast, AMQP flow control credits do not represent bytes. Instead, AMQP credits represent the number of messages a producer is permitted to send, regardless of message size. Therefore, it is possible, in some situations, for AMQP producers to significantly exceed the max-size-bytes value of an address.

Therefore, to block AMQP producers, you must use a different configuration element, max-size-bytes-reject-threshold. For a matching address or set of addresses, this element specifies the maximum size, in bytes, of all AMQP messages in memory. When the total size of all messages in memory reaches the specified limit, the broker blocks AMQP producers from sending further messages.

The following procedure shows how to configure message blocking for AMQP message producers.

Prerequisites

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.

  2. For an address-setting element that you have configured for a matching address or set of addresses, specify the maximum size of all AMQP messages in memory. For example: 

    <address-settings>
    <address-setting match="my.amqp.blocking.address">
        ...
        <max-size-bytes-reject-threshold>300000</max-size-bytes-reject-threshold>
        ...
    </address-setting>
</address-settings>
    max-size-bytes-reject-threshold

    Maximum size, in bytes, of the memory allowed for the address before the broker blocks further AMQP messages. The value that you specify also supports byte notation such as "K", "MB", and "GB". By default, max-size-bytes-reject-threshold is set to -1, which means that there is no maximum size.

    Note

    If you specify max-size-bytes-reject-threshold in an address-setting element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value of max-size-bytes-reject-threshold.

In the preceding example, when messages sent to the address my.amqp.blocking.address exceed 300000 bytes in memory, the broker begins blocking further messages from AMQP producers.

7.4. Understanding memory usage on multicast addresses

When a message is routed to an address that has multicast queues bound to it, there is only one copy of the message in memory. Each queue has only a reference to the message. Because of this, the associated memory is released only after all queues referencing the message have delivered it.

In this type of situation, if you have a slow consumer, the entire address might experience a negative performance impact.

For example, consider this scenario:

  • An address has ten queues that use the multicast routing type.
  • Due to a slow consumer, one of the queues does not deliver its messages. The other nine queues continue to deliver messages and are empty.
  • Messages continue to arrive to the address. The queue with the slow consumer continues to accumulate references to the messages, causing the broker to keep the messages in memory.
  • When the maximum size of the address is reached, the broker starts to page messages.

In this scenario because of a single slow consumer, consumers on all queues are forced to consume messages from the page system, requiring additional IO.

Additional resources

  • To learn how to configure flow control to regulate the flow of data between the broker and producers and consumers, see Flow control in the AMQ Core Protocol JMS documentation.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.