Chapter 7. Configuring maximum memory usage for addresses


AMQ Broker transparently supports huge queues containing millions of messages, even if the machine that is hosting the broker is running with limited memory.

In these situations, it might be not possible to store all of the queues in memory at any one time. To protect against excess memory consumption, you can configure the maximum memory usage that is allowed for each address on the broker. In addition, you can specify what action the broker takes when this limit is reached for a given address.

In particular, when memory usage for an address reaches the configured limit, you can configure the broker to take one of the following actions:

  • Page messages
  • Silently drop messages
  • Drop messages and notify the sending clients
  • Block clients from sending messages

The sections that follow show how to configure maximum memory usage for addresses and the corresponding actions that the broker can take when the limit for an address is reached.

Important

When you use transactions, the broker might allocate extra memory to ensure transactional consistency. In this case, the memory usage reported by the broker might not reflect the total number of bytes being used in memory. Therefore, if you configure the broker to page, drop, or block messages based on a specified maximum memory usage, you should not also use transactions.

7.1. Configuring message paging

For any address that has a maximum memory usage limit specified, you can also specify what action the broker takes when that usage limit is reached. One of the options that you can configure is paging.

If you configure the paging option, when the maximum size of an address is reached, the broker starts to store messages for that address on disk, in files known as page files. Each page file has a maximum size that you can configure. Each address that you configure in this way has a dedicated folder in your file system to store paged messages.

Both queue browsers and consumers can navigate through page files when inspecting messages in a queue. However, a consumer that is using a very specific filter might not be able to consume a message that is stored in a page file until existing messages in the queue have been consumed first. For example, suppose that a consumer filter includes a string expression such as "color='red'". If a message that meets this condition follows one million messages with the property "color='blue'", the consumer cannot consume the message until those with "color='blue'" have been consumed first.

The broker transfers (that is, depages) messages from disk into memory when clients are ready to consume them. The broker removes a page file from disk when all messages in that file have been acknowledged.

The procedures that follow show how to configure message paging.

7.1.1. Specifying a paging directory

The following procedure shows how to specify the location of the paging directory.

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.
  2. Within the core element, add the paging-directory element. Specify a location for the paging directory in your file system.

    <configuration ...>
      <core ...>
        ...
        <paging-directory>/path/to/paging-directory</paging-directory>
        ...
      </core>
    </configuration>

    For each address that you subsequently configure for paging, the broker adds a dedicated directory within the paging directory that you have specified.

7.1.2. Configuring an address for paging

The following procedure shows how to configure an address for paging.

Prerequisites

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.
  2. For an address-setting element that you have configured for a matching address or set of addresses, add configuration elements to specify maximum memory usage and define paging behavior. For example:

    <address-settings>
        <address-setting match="my.paged.address">
            ...
            <max-size-bytes>104857600</max-size-bytes>
            <page-size-bytes>10485760</page-size-bytes>
            <address-full-policy>PAGE</address-full-policy>
            ...
        </address-setting>
    </address-settings>
    max-size-bytes
    Maximum size, in bytes, of the memory allowed for the address before the broker executes the policy specified for address-full-policy. The default value is -1, which means that there is no limit. The value that you specify also supports byte notation such as "K", "MB", and "GB".
    page-size-bytes
    Size, in bytes, of each page file used on the paging system. The default value is 10485760 (that is, 10 MiB). The value that you specify also supports byte notation such as "K", "MB", and "GB".
    address-full-policy

    Action that the broker takes when then the maximum size for an address has been reached. The default value is PAGE. Valid values are:

    PAGE
    The broker pages any further messages to disk.
    DROP
    The broker silently drops any further messages.
    FAIL
    The broker drops any further messages and issues exceptions to client message producers.
    BLOCK
    Client message producers block when they try to send further messages.

    Additional paging configuration elements that are not shown in the preceding example are described below.

    page-max-cache-size
    Number of page files that the broker keeps in memory to optimize IO during paging navigation. The default value is 5.
    page-sync-timeout
    Time, in nanoseconds, between periodic page synchronizations. If you are using an asynchronous IO journal (that is, journal-type is set to ASYNCIO in the broker.xml configuration file), the default value is 3333333. If you are using a standard Java NIO journal (that is, journal-type is set to NIO), the default value is the configured value of the journal-buffer-timeout parameter.

In the preceding example , when messages sent to the address my.paged.address exceed 104857600 bytes in memory, the broker begins paging.

Note

If you specify max-size-bytes in an address-setting element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value of max-size-bytes.

7.1.3. Configuring a global paging size

Sometimes, configuring a memory limit per address is not practical, for example, when a broker manages many addresses that have different usage patterns. In these situations, you can specify a global memory limit. The global limit is the total amount of memory that the broker can use for all addresses. When this memory limit is reached, the broker executes the policy specified for address-full-policy for the address associated with a new incoming message.

The following procedure shows how to configure a global paging size.

Prerequisites

Procedure

  1. Stop the broker.

    1. On Linux:

      <broker_instance_dir>/bin/artemis stop
    2. On Windows:

      <broker_instance_dir>\bin\artemis-service.exe stop
  2. Open the <broker_instance_dir>/etc/broker.xml configuration file.
  3. Within the core element, add the global-max-size element and specify a value. For example:

    <configuration>
      <core>
        ...
        <global-max-size>1GB</global-max-size>
        ...
      </core>
    </configuration>
    global-max-size

    Total amount of memory, in bytes, that the broker can use for all addresses. When this limit is reached, for the address associated with an incoming message, the broker executes the policy that is specified as a value for address-full-policy. The default value of global-max-size is half of the maximum memory available to the Java virtual machine (JVM) that is hosting the broker.

    The value for global-max-size is in bytes, but also supports byte notation (for example, "K", "Mb", "GB").

    In the preceding example, the broker is configured to use a maximum of one gigabyte of available memory when processing messages.

  4. Start the broker.

    1. On Linux:

      <broker_instance_dir>/bin/artemis run
    2. On Windows:

      <broker_instance_dir>\bin\artemis-service.exe start

7.1.4. Limiting disk usage during paging

You can limit the amount of physical disk space that the broker can use before it blocks incoming messages rather than paging them.

The following procedure shows how to set a limit for disk usage during paging.

Procedure

  1. Stop the broker.

    1. On Linux:

      <broker_instance_dir>/bin/artemis stop
    2. On Windows:

      <broker_instance_dir>\bin\artemis-service.exe stop
  2. Open the <broker_instance_dir>/etc/broker.xml configuration file.
  3. Within the core element add the max-disk-usage configuration element and specify a value. For example:

    <configuration>
      <core>
        ...
        <max-disk-usage>50</max-disk-usage>
        ...
      </core>
    </configuration>
    max-disk-usage

    Maximum percentage of the available disk space that the broker can use when paging messages. When this limit is reached, the broker blocks incoming messages rather than paging them. The default value is 90.

    In the preceding example, the broker is limited to using fifty percent of disk space when paging messages.

  4. Start the broker.

    1. On Linux:

      <broker_instance_dir>/bin/artemis run
    2. On Windows:

      <broker_instance_dir>\bin\artemis-service.exe start

7.2. Configuring message dropping

Section 7.1.2, “Configuring an address for paging” shows how to configure an address for paging. As part of that procedure, you set the value of address-full-policy to PAGE.

To drop messages (rather than paging them) when an address reaches its specified maximum size, set the value of the address-full-policy to one of the following:

DROP
When the maximum size of a given address has been reached, the broker silently drops any further messages.
FAIL
When the maximum size of a given address has been reached, the broker drops any further messages and issues exceptions to producers.

7.3. Configuring message blocking

The following procedures show how to configure message blocking when a given address reaches the maximum size limit that you have specified.

Note

You can configure message blocking only for the Core, OpenWire, and AMQP protocols.

7.3.1. Blocking Core and OpenWire producers

The following procedure shows how to configure message blocking for Core and OpenWire message producers when a given address reaches the maximum size limit that you have specified.

Prerequisites

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.
  2. For an address-setting element that you have configured for a matching address or set of addresses, add configuration elements to define message blocking behavior. For example:

    <address-settings>
        <address-setting match="my.blocking.address">
            ...
            <max-size-bytes>300000</max-size-bytes>
            <address-full-policy>BLOCK</address-full-policy>
            ...
        </address-setting>
    </address-settings>
    max-size-bytes

    Maximum size, in bytes, of the memory allowed for the address before the broker executes the policy specified for address-full-policy. The value that you specify also supports byte notation such as "K", "MB", and "GB".

    Note

    If you specify max-size-bytes in an address-setting element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value of max-size-bytes.

    address-full-policy
    Action that the broker takes when then the maximum size for an address has been reached.

In the preceding example, when messages sent to the address my.blocking.address exceed 300000 bytes in memory, the broker begins blocking further messages from Core or OpenWire message producers.

7.3.2. Blocking AMQP producers

Protocols such as Core and OpenWire use a window-size flow control system. In this system, credits represent bytes and are allocated to producers. If a producer wants to send a message, the producer must wait until it has sufficient credits for the size of the message.

By contrast, AMQP flow control credits do not represent bytes. Instead, AMQP credits represent the number of messages a producer is permitted to send, regardless of message size. Therefore, it is possible, in some situations, for AMQP producers to significantly exceed the max-size-bytes value of an address.

Therefore, to block AMQP producers, you must use a different configuration element, max-size-bytes-reject-threshold. For a matching address or set of addresses, this element specifies the maximum size, in bytes, of all AMQP messages in memory. When the total size of all messages in memory reaches the specified limit, the broker blocks AMQP producers from sending further messages.

The following procedure shows how to configure message blocking for AMQP message producers.

Prerequisites

Procedure

  1. Open the <broker_instance_dir>/etc/broker.xml configuration file.
  2. For an address-setting element that you have configured for a matching address or set of addresses, specify the maximum size of all AMQP messages in memory. For example:

    <address-settings>
        <address-setting match="my.amqp.blocking.address">
            ...
            <max-size-bytes-reject-threshold>300000</max-size-bytes-reject-threshold>
            ...
        </address-setting>
    </address-settings>
    max-size-bytes-reject-threshold

    Maximum size, in bytes, of the memory allowed for the address before the broker blocks further AMQP messages. The value that you specify also supports byte notation such as "K", "MB", and "GB". By default, max-size-bytes-reject-threshold is set to -1, which means that there is no maximum size.

    Note

    If you specify max-size-bytes-reject-threshold in an address-setting element, the value applies to each matching address. Specifying this value does not mean that the total size of all matching addresses is limited to the value of max-size-bytes-reject-threshold.

In the preceding example, when messages sent to the address my.amqp.blocking.address exceed 300000 bytes in memory, the broker begins blocking further messages from AMQP producers.

7.4. Understanding memory usage on multicast addresses

When a message is routed to an address that has multicast queues bound to it, there is only one copy of the message in memory. Each queue has only a reference to the message. Because of this, the associated memory is released only after all queues referencing the message have delivered it.

In this type of situation, if you have a slow consumer, the entire address might experience a negative performance impact.

For example, consider this scenario:

  • An address has ten queues that use the multicast routing type.
  • Due to a slow consumer, one of the queues does not deliver its messages. The other nine queues continue to deliver messages and are empty.
  • Messages continue to arrive to the address. The queue with the slow consumer continues to accumulate references to the messages, causing the broker to keep the messages in memory.
  • When the maximum size of the address is reached, the broker starts to page messages.

In this scenario because of a single slow consumer, consumers on all queues are forced to consume messages from the page system, requiring additional IO.

Additional resources

  • To learn how to configure flow control to regulate the flow of data between the broker and producers and consumers, see Flow control in the AMQ Core Protocol JMS documentation.
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.