このコンテンツは選択した言語では利用できません。
Chapter 7. Persisting Messages
This chapter describes how persistence works with AMQ Broker and how to configure it.
The broker ships with two persistence options:
The default. A highly performant option that writes messages to journals on the file system.
Uses the broker’s JDBC Store to persist messages to a database of your choice.
Alternatively, you can also configure the broker for zero persistence.
The broker uses a different solution for persisting large messages outside the message journal. See Working with Large Messages for more information. The broker can also be configured to page messages to disk in low memory situations. See Paging Messages for more information.
For current information regarding which databases and network file systems are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.
7.1. About Journal-based Persistence
A broker’s journal is a set of append only files on disk. Each file is pre-created to a fixed size and initially filled with padding. As messaging operations are performed on the broker, records are appended to end of the journal. Appending records allows the broker to minimize disk head movement and random access operations, which are typically the slowest operation on a disk. When one journal file is full, the broker uses a new one.
The journal file size is configurable, minimizing the number of disk cylinders used by each file. Modern disk topologies are complex, however, and the broker cannot control which cylinder(s) the file is mapped to. Journal file sizing therefore is not an exact science.
Other persistence-related features include:
- A sophisticated file garbage collection algorithm that determines whether a particular journal file is still in use. If not, the file can be reclaimed and re-used.
- A compaction algorithm that removes dead space from the journal and that compresses the data. This results in the journal using fewer files on disk.
- Support for local transactions.
- Support for XA transactions when using AMQ JMS clients.
The majority of the journal is written in Java. However, the interaction with the actual file system is abstracted, so you can use different, pluggable implementations. AMQ Broker ships with two implementations:
Uses the standard Java NIO to interface with the file system. This provides extremely good performance and runs on any platform with a Java 6 or later runtime.
Linux Asynchronous IO
Uses a thin native wrapper to talk to the Linux asynchronous IO library (AIO). With AIO, the broker is called back after the data has made it to disk, avoiding explicit syncs altogether. By default the broker tries to use an AIO journal, and falls back to using NIO if AIO is not available.
Using AIO typically provides even better performance than using Java NIO. For instructions on how to install libaio see Using an AIO journal.
For current information regarding which network file systems are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.
7.1.1. Using AIO
The Java NIO journal is highly performant, but if you are running the broker using Linux Kernel 2.6 or later, Red Hat recommends using the AIO journal for better persistence performance. It is not possible to use the AIO journal with other operating systems or earlier versions of the Linux kernel.
To use the AIO journal you must install the libaio if it is not already installed.
Procedure
-
Use the
yumcommand to installlibaio, as in the example below:
yum install libaio
7.2. Configuring Journal-based Persistence
Persistence configuration is maintained in the file BROKER_INSTANCE_DIR/etc/broker.xml. The broker’s default configuration uses journal based persistence and includes the elements shown below.
<configuration>
<core>
...
<persistence-enabled>true</persistence-enabled> 1
<journal-type>ASYNCIO</journal-type> 2
<bindings-directory>./data/bindings</bindings-directory> 3
<journal-directory>./data/journal</journal-directory> 4
<journal-datasync>true</journal-datasync> 5
<journal-min-files>2</journal-min-files> 6
<journal-pool-files>-1</journal-pool-files> 7
...
</core>
</configuration>- 1
- Set to
trueto use the file based journal for persistence. - 2
- The type of journal to use. If set to ASYNCIO, the broker first attempts to use AIO and falls back to NIO if ASYNCIO is not found.
- 3
- The file system location of the bindings journal. The default setting is relative to
BROKER_INSTANCE_DIR. - 4
- The file system location of the messaging journal. The default setting is relative to
BROKER_INSTANCE_DIR. - 5
- Set to
trueto usefdatasyncto confirm writes to the disk. - 6
- The number of journal files to pre-create when the broker starts.
- 7
- The number of files to keep after reclaiming un-used files. The default value, -1, means that no files are deleted during clean up.
7.2.1. The Message Journal
The message journal stores all message-related data, including the messages themselves and duplicate ID caches. The files on this journal are prefixed as activemq-data. Each file has a amq extension and a default size of 10485760 bytes. The location of the message journal is set using the journal-directory configuration element. The default value is BROKER_INSTANCE_DIR/data/journal. The default configuration includes other elements related to the messaging journal:
journal-min-filesThe number of journal files to pre-create when the broker starts. The default is
2.journal-pool-filesThe number of files to keep after reclaiming un-used files. The default value,
-1,means that no files are deleted once created by the broker. However, the system cannot grow infinitely, so you are required to use paging for destinations that are unbounded in this way. See the chapter on Paging Messages for more information.
There are several other configuration elements available for the messaging journal. See the appendix for a full list.
7.2.2. The Bindings Journal
The bindings journal is used to store bindings-related data, such as the set of queues deployed on the server and their attributes. It also stores data such as ID sequence counters.
The bindings journal always uses NIO because it is typically low throughput when compared to the message journal. Files on this journal are prefixed with activemq-bindings. Each file has a bindings extension and a default size of 1048576 bytes.
Use the following configuration elements in BROKER_INSTANCE_DIR/etc/broker.xml to configure the bindings journal.
bindings-directoryThis is the directory in which the bindings journal lives. The default value is
BROKER_INSTANCE_DIR/data/bindings.create-bindings-dirIf this is set to
truethen the bindings directory is automatically created at the location specified inbindings-directoryif it does not already exist. The default value istrue
7.2.3. The JMS Journal
The JMS journal stores all JMS-related data, including JMS Queues, Topics, and Connection Factories, as well as any JNDI bindings for these resources. Also, any JMS Resources created via the management API is persisted to this journal, but any resources configured via configuration files are not. The JMS Journal is only created if JMS is being used.
The files on this journal are prefixed as activemq-jms. Each file has a jms extension and and a default size of 1048576 bytes.
The JMS journal shares its configuration with the bindings journal.
7.2.4. Compacting Journal Files
AMQ Broker includes a compaction algorithm that removes dead space from the journal and compresses its data so that it takes up less space on disk. There are two criteria used to determine when to start compaction. After both criteria are met, the compaction process parses the journal and removes all dead records. Consequently, the journal comprises fewer files. The criteria are:
- The number of files created for the journal.
- The percentage of live data in the journal’s files.
You configure both criteria in BROKER_INSTANCE_DIR/etc/broker.xml.
Procedure
To configure the criteria for the compaction process, add the following two elements, as in the example below.
<configuration> <core> ... <journal-compact-min-files>15</journal-compact-min-files> 1 <journal-compact-percentage>25</journal-compact-percentage> 2 ... </core> </configuration>- 1
- The minimum number of files created before compaction begins. That is, the compacting algorithm does not start until you have at least
journal-compact-min-files. The default value is10. Setting this to0disables compaction, which is dangerous because the journal could grow indefinitely. - 2
- The percentage of live data in the journal’s files. When less than this percentage is considered live data, compacting begins. Remember that compacting does not begin until you also have at least
journal-compact-min-filesdata files on the journal. The default value is30.
Compacting Journals Using the CLI
You can also use the command-line interface (CLI) to compact journals.
Procedure
As the owner of the
BROKER_INSTANCE_DIR, stop the broker. In the example below, the useramq-brokerwas created during the installation of AMQ Broker.su - amq-broker cd __BROKER_INSTANCE_DIR__/bin $ ./artemis stop
(Optional) Run the following CLI command to get a full list of parameters for the data tool. Note that by default, the tool uses settings found in
BROKER_INSTANCE_DIR/etc/broker.xml.$ ./artemis help data compact.
Run the following CLI command to compact the data.
$ ./artemis data compact.
After the tool has successfully compacted the data, restart the broker.
$ ./artemis run
Related Information
AMQ Broker includes a number of CLI commands for managing your journal files. See command-line Tools in the Appendix for more information.
7.2.5. Disabling Disk Write Cache
Most disks contain hardware write caches. A write cache can increase the apparent performance of the disk because writes are lazily written to the disk later. By default many systems ship with disk write cache enabled. This means that even after syncing from the operating system there is no guarantee the data has actually made it to disk, so if a failure occurs, critical data can be lost.
Some more expensive disks have non-volatile or battery-backed write caches that do not necessarily lose data in event of failure, but you should test them. If your disk does not have such features, you should ensure that write cache is disabled. Be aware that disabling disk write cache can negatively affect performance.
Procedure
-
On Linux, manage your disk’s write cache settings using the tools
hdparm(for IDE disks) orsdparmorsginfo(for SDSI/SATA disks). - On Windows, manage the cache setting by right-clicking the disk and clicking Properties.
7.3. Configuring JDBC Persistence
The JDBC persistence store uses a JDBC connection to store messages and bindings data in database tables. The data in the tables is encoded using AMQ Broker journal encoding.
For current information regarding which databases are supported see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal.
Procedure
-
Add the appropriate JDBC client libraries to the broker runtime. You can do this by adding the relevant jars to the
BROKER_INSTANCE_DIR/libdirectory. Create a
storeelement in yourBROKER_INSTANCE_DIR/etc/broker.xmlconfiguration file under thecoreelement, as in the example below.<configuration> <core> <store> <database-store> <jdbc-connection-url>jdbc:derby:data/derby/database-store;create=true</jdbc-connection-url> 1 <bindings-table-name>BINDINGS_TABLE</bindings-table-name> 2 <message-table-name>MESSAGE_TABLE</message-table-name> 3 <large-message-table-name>LARGE_MESSAGES_TABLE</large-message-table-name> 4 <jdbc-driver-class-name>org.apache.derby.jdbc.EmbeddedDriver</jdbc-driver-class-name> 5 </database-store> </store> </core> </configuration>- 1
jdbc-connection-urlis the full JDBC connection URL for your database server. The connection url should include all configuration parameters and database name.- 2
bindings-table-nameis the name of the table in which the bindings data is stored. Specifying table name allows users to share single database amongst multiple servers, without interference.- 3
message-table-nameis the name of the table in which the bindings data is stored.- 4
large-message-table-nameis the name of the table in which messages and related data is persisted.- 5
jdbc-driver-class-nameis the fully qualified class name of the desired database Driver.
7.4. Configuring Zero Persistence
In some situations, zero persistence is sometimes required for a messaging system. Configuring the broker to perform zero persistence is straightforward. Set the parameter persistence-enabled in BROKER_INSTANCE_DIR/etc/broker.xml to false.
Note that if you set this parameter to false, then zero persistence occurs. That means no bindings data, message data, large message data, duplicate ID caches or paging data is persisted.
