ApoyoConsolaDesarrolladoresIniciar una pruebaContacto

Este contenido no está disponible en el idioma seleccionado.

Chapter 7. Persisting Messages

This chapter describes how persistence works with AMQ Broker and how to configure it.

The broker ships with two persistence options:

  1. Journal-based

    The default. A highly performant option that writes messages to journals on the file system.

  2. JDBC-based

    Uses the broker’s JDBC Store to persist messages to a database of your choice.

Alternatively, you can also configure the broker for zero persistence.

The broker uses a different solution for persisting large messages outside the message journal. See Working with Large Messages for more information. The broker can also be configured to page messages to disk in low memory situations. See Paging Messages for more information.

Note

For current information regarding which databases and network file systems are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.

7.1. About Journal-based Persistence

A broker’s journal is a set of append only files on disk. Each file is pre-created to a fixed size and initially filled with padding. As messaging operations are performed on the broker, records are appended to end of the journal. Appending records allows the broker to minimize disk head movement and random access operations, which are typically the slowest operation on a disk. When one journal file is full, the broker uses a new one.

The journal file size is configurable, minimizing the number of disk cylinders used by each file. Modern disk topologies are complex, however, and the broker cannot control which cylinder(s) the file is mapped to. Journal file sizing therefore is not an exact science.

Other persistence-related features include:

  • A sophisticated file garbage collection algorithm that determines whether a particular journal file is still in use. If not, the file can be reclaimed and re-used.
  • A compaction algorithm that removes dead space from the journal and that compresses the data. This results in the journal using fewer files on disk.
  • Support for local transactions.
  • Support for XA transactions when using AMQ JMS clients.

The majority of the journal is written in Java. However, the interaction with the actual file system is abstracted, so you can use different, pluggable implementations. AMQ Broker ships with two implementations:

  • Java NIO.

    Uses the standard Java NIO to interface with the file system. This provides extremely good performance and runs on any platform with a Java 6 or later runtime.

  • Linux Asynchronous IO

    Uses a thin native wrapper to talk to the Linux asynchronous IO library (AIO). With AIO, the broker is called back after the data has made it to disk, avoiding explicit syncs altogether. By default the broker tries to use an AIO journal, and falls back to using NIO if AIO is not available.

    Using AIO typically provides even better performance than using Java NIO. For instructions on how to install libaio see Using an AIO journal.

Note

For current information regarding which network file systems are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.

7.1.1. Using AIO

The Java NIO journal is highly performant, but if you are running the broker using Linux Kernel 2.6 or later, Red Hat recommends using the AIO journal for better persistence performance. It is not possible to use the AIO journal with other operating systems or earlier versions of the Linux kernel.

To use the AIO journal you must install the libaio if it is not already installed.

Procedure

  • Use the yum command to install libaio, as in the example below: 
yum install libaio

7.2. Configuring Journal-based Persistence

Persistence configuration is maintained in the file BROKER_INSTANCE_DIR/etc/broker.xml. The broker’s default configuration uses journal based persistence and includes the elements shown below. 

<configuration>
  <core>
    ...
    <persistence-enabled>true</persistence-enabled> 1
    <journal-type>ASYNCIO</journal-type>  2
    <bindings-directory>./data/bindings</bindings-directory>  3
    <journal-directory>./data/journal</journal-directory>  4
    <journal-datasync>true</journal-datasync>  5
    <journal-min-files>2</journal-min-files>  6
    <journal-pool-files>-1</journal-pool-files>  7
    ...
  </core>
</configuration>
1
Set to true to use the file based journal for persistence.
2
The type of journal to use. If set to ASYNCIO, the broker first attempts to use AIO and falls back to NIO if ASYNCIO is not found.
3
The file system location of the bindings journal. The default setting is relative to BROKER_INSTANCE_DIR.
4
The file system location of the messaging journal. The default setting is relative to BROKER_INSTANCE_DIR.
5
Set to true to use fdatasync to confirm writes to the disk.
6
The number of journal files to pre-create when the broker starts.
7
The number of files to keep after reclaiming un-used files. The default value, -1, means that no files are deleted during clean up.

7.2.1. The Message Journal

The message journal stores all message-related data, including the messages themselves and duplicate ID caches. The files on this journal are prefixed as activemq-data. Each file has a amq extension and a default size of 10485760 bytes. The location of the message journal is set using the journal-directory configuration element. The default value is BROKER_INSTANCE_DIR/data/journal. The default configuration includes other elements related to the messaging journal:

  • journal-min-files

    The number of journal files to pre-create when the broker starts. The default is 2.

  • journal-pool-files

    The number of files to keep after reclaiming un-used files. The default value, -1, means that no files are deleted once created by the broker. However, the system cannot grow infinitely, so you are required to use paging for destinations that are unbounded in this way. See the chapter on Paging Messages for more information.

There are several other configuration elements available for the messaging journal. See the appendix for a full list.

7.2.2. The Bindings Journal

The bindings journal is used to store bindings-related data, such as the set of queues deployed on the server and their attributes. It also stores data such as ID sequence counters.

The bindings journal always uses NIO because it is typically low throughput when compared to the message journal. Files on this journal are prefixed with activemq-bindings. Each file has a bindings extension and a default size of 1048576 bytes.

Use the following configuration elements in BROKER_INSTANCE_DIR/etc/broker.xml to configure the bindings journal.

  • bindings-directory

    This is the directory in which the bindings journal lives. The default value is BROKER_INSTANCE_DIR/data/bindings.

  • create-bindings-dir

    If this is set to true then the bindings directory is automatically created at the location specified in bindings-directory if it does not already exist. The default value is true

7.2.3. The JMS Journal

The JMS journal stores all JMS-related data, including JMS Queues, Topics, and Connection Factories, as well as any JNDI bindings for these resources. Also, any JMS Resources created via the management API is persisted to this journal, but any resources configured via configuration files are not. The JMS Journal is only created if JMS is being used.

The files on this journal are prefixed as activemq-jms. Each file has a jms extension and and a default size of 1048576 bytes.

The JMS journal shares its configuration with the bindings journal.

7.2.4. Compacting Journal Files

AMQ Broker includes a compaction algorithm that removes dead space from the journal and compresses its data so that it takes up less space on disk. There are two criteria used to determine when to start compaction. After both criteria are met, the compaction process parses the journal and removes all dead records. Consequently, the journal comprises fewer files. The criteria are:

  • The number of files created for the journal.
  • The percentage of live data in the journal’s files.

You configure both criteria in BROKER_INSTANCE_DIR/etc/broker.xml.

Procedure

  • To configure the criteria for the compaction process, add the following two elements, as in the example below. 

    <configuration>
  <core>
    ...
    <journal-compact-min-files>15</journal-compact-min-files> 1
    <journal-compact-percentage>25</journal-compact-percentage>  2
    ...
  </core>
</configuration>
    1
    The minimum number of files created before compaction begins. That is, the compacting algorithm does not start until you have at least journal-compact-min-files. The default value is 10. Setting this to 0 disables compaction, which is dangerous because the journal could grow indefinitely.
    2
    The percentage of live data in the journal’s files. When less than this percentage is considered live data, compacting begins. Remember that compacting does not begin until you also have at least journal-compact-min-files data files on the journal. The default value is 30.
Compacting Journals Using the CLI

You can also use the command-line interface (CLI) to compact journals.

Procedure

  1. As the owner of the BROKER_INSTANCE_DIR, stop the broker. In the example below, the user amq-broker was created during the installation of AMQ Broker. 

    su - amq-broker
cd __BROKER_INSTANCE_DIR__/bin
$ ./artemis stop

  2. (Optional) Run the following CLI command to get a full list of parameters for the data tool. Note that by default, the tool uses settings found in BROKER_INSTANCE_DIR/etc/broker.xml. 

    $ ./artemis help data compact.

  3. Run the following CLI command to compact the data. 

    $ ./artemis data compact.

  4. After the tool has successfully compacted the data, restart the broker. 

    $ ./artemis run

Related Information

AMQ Broker includes a number of CLI commands for managing your journal files. See command-line Tools in the Appendix for more information.

7.2.5. Disabling Disk Write Cache

Most disks contain hardware write caches. A write cache can increase the apparent performance of the disk because writes are lazily written to the disk later. By default many systems ship with disk write cache enabled. This means that even after syncing from the operating system there is no guarantee the data has actually made it to disk, so if a failure occurs, critical data can be lost.

Some more expensive disks have non-volatile or battery-backed write caches that do not necessarily lose data in event of failure, but you should test them. If your disk does not have such features, you should ensure that write cache is disabled. Be aware that disabling disk write cache can negatively affect performance.

Procedure

  • On Linux, manage your disk’s write cache settings using the tools hdparm (for IDE disks) or sdparm or sginfo (for SDSI/SATA disks).
  • On Windows, manage the cache setting by right-clicking the disk and clicking Properties.

7.3. Configuring JDBC Persistence

The JDBC persistence store uses a JDBC connection to store messages and bindings data in database tables. The data in the tables is encoded using AMQ Broker journal encoding.

Note

For current information regarding which databases are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.

Procedure

  1. Add the appropriate JDBC client libraries to the broker runtime. You can do this by adding the relevant jars to the BROKER_INSTANCE_DIR/lib directory.

  2. Create a store element in your BROKER_INSTANCE_DIR/etc/broker.xml configuration file under the core element, as in the example below. 

    <configuration>
  <core>
    <store>
       <database-store>
          <jdbc-connection-url>jdbc:derby:data/derby/database-store;create=true</jdbc-connection-url> 1
          <bindings-table-name>BINDINGS_TABLE</bindings-table-name> 2
          <message-table-name>MESSAGE_TABLE</message-table-name> 3
          <large-message-table-name>LARGE_MESSAGES_TABLE</large-message-table-name> 4
          <jdbc-driver-class-name>org.apache.derby.jdbc.EmbeddedDriver</jdbc-driver-class-name> 5
       </database-store>
    </store>
  </core>
</configuration>
    1
    jdbc-connection-url is the full JDBC connection URL for your database server. The connection url should include all configuration parameters and database name.
    2
    bindings-table-name is the name of the table in which the bindings data is stored. Specifying table name allows users to share single database amongst multiple servers, without interference.
    3
    message-table-name is the name of the table in which the bindings data is stored.
    4
    large-message-table-name is the name of the table in which messages and related data is persisted.
    5
    jdbc-driver-class-name is the fully qualified class name of the desired database Driver.

7.4. Configuring Zero Persistence

In some situations, zero persistence is sometimes required for a messaging system. Configuring the broker to perform zero persistence is straightforward. Set the parameter persistence-enabled in BROKER_INSTANCE_DIR/etc/broker.xml to false.

Note that if you set this parameter to false, then zero persistence occurs. That means no bindings data, message data, large message data, duplicate ID caches or paging data is persisted.

Red Hat logoGithubRedditYoutubeTwitter

Aprender

Pruebe, compre y venda

Comunidades

Acerca de la documentación de Red Hat

Ayudamos a los usuarios de Red Hat a innovar y alcanzar sus objetivos con nuestros productos y servicios con contenido en el que pueden confiar.

Hacer que el código abierto sea más inclusivo

Red Hat se compromete a reemplazar el lenguaje problemático en nuestro código, documentación y propiedades web. Para más detalles, consulte el Blog de Red Hat.

Acerca de Red Hat

Ofrecemos soluciones reforzadas que facilitan a las empresas trabajar en plataformas y entornos, desde el centro de datos central hasta el perímetro de la red.

© 2024 Red Hat, Inc.