Ce contenu n'est pas disponible dans la langue sélectionnée.
Chapter 10. Configuring Persistence
10.1. About Persistence in JBoss EAP 7 Messaging
JBoss EAP messaging handles its own persistence. It ships with a high-performance journal, which is optimized for messaging.
The JBoss EAP messaging journal has a configurable file size and is append only, which improves performance by enabling single write operations. It consists of a set of files on disk, which are initially pre-created to a fixed size and filled with padding. As server operations (add message, delete message, update message, etc.) are performed, records of the operations are appended to the journal until the journal file is full, at which point the next journal file is used.
A sophisticated garbage collection algorithm determines whether journal files can be reclaimed and re-used when all of their data has been deleted. A compaction algorithm removes dead space from journal files and compresses the data.
The journal also fully supports both local and XA transactions.
The majority of the journal is written in Java, but interaction with the file system has been abstracted to allow different pluggable implementations. The two implementations shipped with JBoss EAP messaging are:
- Java New I/O (NIO)
- This implementation uses standard Java NIO to interface with the file system. It provides extremely good performance and runs on any platform with a Java 6 or later runtime. Note that JBoss EAP 7 requires Java 8. Using NIO is supported on any operating system that JBoss EAP supports.
- Linux Asynchronous IO (ASYNCIO)
This implementation uses a native code wrapper to talk to the Linux asynchronous IO library (ASYNCIO). This implementation removes the need for explicit synchronization. ASYNCIO typically provides better performance than Java NIO.
The following file systems have been tested and are supported only on Red Hat Enterprise Linux 6 and Red Hat Enterprise Linux 7 when using the
libaio
natives. They are not tested and are not supported on other operating systems.- EXT4
- XFS
- NFSv4
- GFS2
The following table lists the HA shared store file systems that have been tested, both with and without the
libaio
natives, and whether they are supported.Operating System File System Supported Using libaio
Natives?
(journal-type="ASYNCIO")Supported Without Using libaio
Natives?
(journal-type="NIO")Red Hat Enterprise Linux 6
NFSv4
Yes
Yes
Red Hat Enterprise Linux 7
NFSv4
Yes
Yes
Red Hat Enterprise Linux 6
GFS2
Yes
No
Red Hat Enterprise Linux 7
GFS2
Yes
No
The standard JBoss EAP messaging core server uses the following journal instances:
- Bindings journal
This journal is used to store bindings related data, including the set of queues that are deployed on the server and their attributes. It also stores data such as id sequence counters.
The bindings journal is always a NIO journal as it is typically low throughput compared to the message journal.
The files on this journal are prefixed as activemq-bindings. Each file has a bindings extension. File size is 1048576, and it is located at the bindings folder.
- JMS journal
This journal instance stores all JMS related data, such as any JMS queues,topics, connection factories and any JNDI bindings for these resources.
Any JMS Resource created via the management API will be persisted to this journal. Any resource configured via configuration files will not. The JMS Journal will only be created if JMS is being used.
The files on this journal are prefixed as activemq-jms. Each file has a jms extension. File size is 1048576, and it is located at the bindings folder.
- Message journal
This journal instance stores all message related data, including the message themselves and also duplicate-id caches.
By default JBoss EAP messaging will try to use an ASYNCIO journal. If ASYNCIO is not available, for example the platform is not Linux with the correct kernel version or ASYNCIO has not been installed then it will automatically fall back to using Java NIO which is available on any Java platform.
The files on this journal are prefixed as activemq-data. Each file has an amq extension. File size is by default 10485760 (configurable), and it is located at the journal folder.
For large messages, JBoss EAP messaging persists them outside the message journal. This is discussed in the section on Large Messages.
JBoss EAP messaging can also be configured to page messages to disk in low memory situations. This is discussed in the Paging section.
If no persistence is required at all, JBoss EAP messaging can also be configured not to persist any data at all to storage as discussed in the Configuring JBoss EAP Messaging for Zero Persistence section.
10.2. Configuring the Bindings and JMS Journals
Because the bindings journal shares its configuration with the JMS journal, you can read the current configuration for both by using the single management CLI command below. The output is also included to highlight default configuration.
/subsystem=messaging-activemq/server=default/path=bindings-directory:read-resource { "outcome" => "success", "result" => { "path" => "activemq/bindings", "relative-to" => "jboss.server.data.dir" } }
Note that by default the path
to the journal is activemq/bindings
. You can change the location for path
by using the following management CLI command.
/subsystem=messaging-activemq/server=default/path=bindings-directory:write-attribute(name=path,value=<PATH_LOCATION>)
Also note the relative-to
attribute in the output above. When relative-to
is used, the value of the path
attribute is treated as relative to the file path specified by relative-to
. By default this value is the JBoss EAP property jboss.server.data.dir
. For standalone servers, jboss.server.data.dir
is located at EAP_HOME/standalone/data
. For domains, each server will have its own serverX/data/activemq
directory located under EAP_HOME/domain/servers
. You can change the value of relative-to
using the following management CLI command.
/subsystem=messaging-activemq/server=default/path=bindings-directory:write-attribute(name=relative-to,value=<RELATIVE_LOCATION>)
By default, JBoss EAP is configured to automatically create the bindings directory if it does not exist. Use the following management CLI command to toggle this behavior.
/subsystem=messaging-activemq/server=default:write-attribute(name=create-bindings-dir,value=<TRUE_FALSE>)
Setting value
to true
will enable automatic directory creation. Setting value
to false
will disable it.
10.3. Configuring the Message Journal Location
You can read the location information for the message journal by using the management CLI command below. The output is also included to highlight default configuration.
/subsystem=messaging-activemq/server=default/path=journal-directory:read-resource { "outcome" => "success", "result" => { "path" => "activemq/journal", "relative-to" => "jboss.server.data.dir" } }
Note that by default the path
to the journal is activemq/journal
. You can change the location for path
by using the following management CLI command.
/subsystem=messaging-activemq/server=default/path=journal-directory:write-attribute(name=path,value=<PATH_LOCATION>)
For the best performance, Red Hat recommends that the journal be located on its own physical volume in order to minimize disk head movement. If the journal is on a volume which is shared with other processes which might be writing other files, such as a bindings journal, database, or transaction coordinator, then the disk head may well be moving rapidly between these files as it writes them, thus drastically reducing performance.
Also note the relative-to
attribute in the output above. When relative-to
is used, the value of the path
attribute is treated as relative to the file path specified by relative-to
. By default this value is the JBoss EAP property jboss.server.data.dir
. For standalone servers, jboss.server.data.dir
is located at EAP_HOME/standalone/data
. For domains, each server will have its own serverX/data/activemq
directory located under EAP_HOME/domain/servers
. You can change the value of relative-to
using the following management CLI command.
/subsystem=messaging-activemq/server=default/path=journal-directory:write-attribute(name=relative-to,value=<RELATIVE_LOCATION>)
By default, JBoss EAP is configured to automatically create the journal directory if it does not exist. Use the following management CLI command to toggle this behavior.
/subsystem=messaging-activemq/server=default:write-attribute(name=create-journal-dir,value=<TRUE_FALSE>)
Setting value
to true
will enable automatic directory creation. Setting value
to false
will disable it.
10.4. Configuring Message Journal Attributes
The attributes listed below are all child properties of the messaging server. Therefore, the command syntax for getting and setting their values using the management CLI is the same for each.
To read the current value of a given attribute, the syntax is as follows:
/subsystem=messaging-activemq/server=default:read-attribute(name=<ATTRIBUTE_NAME>)
The syntax for writing an attribute’s value follows a corresponding pattern.
/subsystem=messaging-activemq/server=default:write-attribute(name=<ATTRIBUTE_NAME>,value=<NEW_VALUE>)
create-journal-dir
If this is set to true, the journal directory will be automatically created at the location specified in
journal-directory
if it does not already exist. The default value istrue
.journal-buffer-timeout
Instead of flushing on every write that requires a flush, we maintain an internal buffer, and flush the entire buffer either when it is full, or when a timeout expires, whichever is sooner. This is used for both NIO and ASYNCIO and allows the system to scale better with many concurrent writes that require flushing.
This parameter controls the timeout at which the buffer will be flushed if it has not filled already. ASYNCIO can typically cope with a higher flush rate than NIO, so the system maintains different defaults for both NIO and ASYNCIO. The default for NIO is
3333333
nanoseconds, or 300 times per second. The default for ASYNCIO is500000
nanoseconds, or 2000 times per second.NoteBy increasing the timeout, you may be able to increase system throughput at the expense of latency, the default parameters are chosen to give a reasonable balance between throughput and latency.
journal-buffer-size
The size, in bytes, of the timed buffer on ASYNCIO. Both
journal-buffer-size
andjournal-file-size
must be set larger thanmin-large-message-size
. Otherwise, messages will not be written to the journal. See Configuring Large Messages for more information.journal-compact-min-files
The minimal number of files before we can consider compacting the journal. The compacting algorithm won’t start until you have at least
journal-compact-min-files
.Setting this to
0
will disable the feature to compact completely. This could be dangerous though as the journal could grow indefinitely. Use it wisely!The default for this parameter is
10
journal-compact-percentage
The threshold to start compacting. When less than this percentage is considered live data, we start compacting. Note also that compacting will not kick in until you have at least
journal-compact-min-files
data files on the journalThe default for this parameter is
30
.journal-file-size
The size of each journal file, in bytes. The default value for this is
10485760
bytes, or 10MB. Bothjournal-file-size
andjournal-buffer-size
must be set larger thanmin-large-message-size
. Otherwise, messages will not be written to the journal. See Configuring Large Messages for more information.journal-max-io
Write requests are queued up before being submitted to the system for execution. This parameter controls the maximum number of write requests that can be in the IO queue at any one time. If the queue becomes full then writes will block until space is freed up.
The system maintains different defaults for this parameter depending on whether it’s NIO or ASYNCIO. The default for NIO is
1
, and the default for ASYNCIO is500
.There is a limit and the total max ASYNCIO cannot be higher than what is configured at the OS level, found at /proc/sys/fs/aio-max-nr, usually
65536
.journal-min-files
The minimum number of files the journal will maintain. When JBoss EAP starts and there is no initial message data, JBoss EAP will pre-create
journal-min-files
number of files. The default is2
.Creating journal files and filling them with padding is a fairly expensive operation and we want to minimize doing this at run-time as files get filled. By pre-creating files, as one is filled the journal can immediately resume with the next one without pausing to create it.
Depending on how much data you expect your queues to contain at steady state you should tune this number of files to match that total amount of data.
journal-pool-files
The number of journal files that can be reused. ActiveMQ will create as many files as needed however when reclaiming files it will shrink back to the value. The default is
-1
, which means no limit.journal-sync-transactional
If this is set to true then JBoss EAP will make sure all transaction data is flushed to disk on transaction boundaries, such as a commit, prepare, or rollback. The default value is
true
.journal-sync-non-transactional
If this is set to true then JBoss EAP will make sure non transactional message data, such as sends and acknowledgements, are flushed to disk each time. The default value is
true
.journal-type
Valid values are
NIO
orASYNCIO
.Choosing
NIO
tells JBoss EAP to use a Java NIO journal.ASYNCIO
tells it to use a Linux asynchronous IO journal. If you chooseASYNCIO
but are not running Linux, or you do not have libaio installed, JBoss EAP will use a Java NIO journal.
10.5. Note on Disabling Disk Write Cache
This happens irrespective of whether you have executed a fsync()
from the operating system or correctly synced data from inside a Java program!
By default many systems ship with disk write cache enabled. This means that even after syncing from the operating system there is no guarantee the data has actually made it to disk, so if a failure occurs, critical data can be lost.
Some more expensive disks have non volatile or battery backed write caches which won’t necessarily lose data on event of failure, but you need to test them!
If your disk does not have an expensive non volatile or battery backed cache and it’s not part of some kind of redundant array (e.g. RAID), and you value your data integrity you need to make sure disk write cache is disabled.
Be aware that disabling disk write cache can give you a nasty shock performance wise. If you’ve been used to using disks with write cache enabled in their default setting, unaware that your data integrity could be compromised, then disabling it will give you an idea of how fast your disk can perform when acting really reliably.
On Linux you can inspect or change your disk’s write cache settings using the tools hdparm
(for IDE disks) or sdparm
or sginfo
(for SDSI/SATA disks)
On Windows you can check / change the setting by right clicking on the disk and clicking properties.
10.6. Installing libaio
The Java NIO journal is highly performant, but if you are running JBoss EAP messaging using Linux Kernel 2.6 or later, Red Hat highly recommends that you use the ASYNCIO journal for the very best persistence performance.
JBoss EAP supports ASYNCIO only when installed on versions 6 or 7 of Red Hat Enterprise Linux and only when using the ext4, xfs, gfs2 or nfs4 file systems. It is not possible to use the ASYNCIO journal under other operating systems or earlier versions of the Linux kernel.
You will need libaio
installed to use the ASYNCIO journal. You can easily install it using the following YUM command:
yum install libaio
Do not place your messaging journals on a tmpfs file system, which is used for the /tmp
directory for example. JBoss EAP will fail to start if the ASYNCIO journal is using tmpfs.
10.8. Configuring JBoss EAP Messaging for Zero Persistence
In some situations, zero persistence is required for a messaging system. Zero persistence means that no bindings data, message data, large message data, duplicate id caches, or paging data should be persisted.
To configure the messaging-activemq
subsystem to perform zero persistence, set the persistence-enabled
parameter to false
.
/subsystem=messaging-activemq/server=default:write-attribute(name=persistence-enabled,value=false)
Be aware that if persistence is disabled, but paging is enabled, page files continue to be stored in the location specified by the paging-directory
element. Paging is enabled when the address-full-policy
attribute is set to PAGE
. If full zero persistence is required, be sure to configure the address-full-policy
attribute of the address-setting
element to use BLOCK
, DROP
or FAIL
.
10.9. Importing and Exporting Journal Data
See the JBoss EAP 7 Migration Guide for information on importing and exporting journal data.