Chapter 38. Colocated and Dedicated Symmetrical Cluster Configuration
An instance of HornetQ running on JBoss Enterprise Application Platform that is configured to fail over to a specified group of HornetQ instances.
- Colocated
- Topology containing one live, and at least one back-up server running concurrently. Each backup node belongs to a live node on another JBoss Enterprise Application Platform instance.
- Dedicated
- Topology containing one live and at least one backup server. Only one server can run at any given time.
38.1. Colocated Symmetrical Live and Backup Cluster
Note
$JBOSS_HOME/extras/hornetq/resources/examples/symmetric-cluster-with-backups-colocated
. The readme
in this directory provides basic configuration required to run the example.
Example 38.1. Two Instance Configuration
Note
Example 38.2. Three Instance Configuration
38.1.1. Colocated Live Server
Important
Procedure 38.1. Create Live Server Profile
production
profile to customize the live server configuration.
Important
- Navigate to
$JBOSS_HOME/server/
- Copy the
production
profile, and rename it toHornetQ_Colocated
Procedure 38.2. Configure Shared Store and Journaling
- Navigate to
$JBOSS_HOME/server/HornetQ_Colocated/deploy/hornetq/
- Open
- Add the <shared-store> element as a child of the <configuration> element.
<shared-store>true</shared-store>
- Ensure the bindings, journal, and large messages path locations are set to a location the live backup group can access.You can set absolute paths as the example describes, or use the JBoss parameters that exist in the configuration file.If you choose the parameter option, and you do not use the default paths that these parameters resolve to, you must specify the path your bindings, journal, and large messages reside in each time you start the server.
<large-messages-directory>/media/shared/data/serverA/large-messages</large-messages-directory> <bindings-directory>/media/shared/data/serverA/bindings</bindings-directory> <journal-directory>/media/shared/data/serverA/journal</journal-directory> <paging-directory>/media/shared/data/ServerA/paging</paging-directory>
Note
Ensure you specify paths that are accessible to the live backup groups on your network.Note
Change ServerA to the name suitable to your server instance.
Procedure 38.3. Configure JMS Client Graceful Shutdown
- Navigate to
$JBOSS_HOME/server/HornetQ_Colocated/deploy/hornetq/
- Open
hornetq-configuration.xml
- Specify the <fail-over-on-shutdown> element in the area near the journal directory configuration in Procedure 38.2, “Configure Shared Store and Journaling”.
<failover-on-shutdown>true</failover-on-shutdown>
Note
You are not constrained where you put the element in thehornetq-configuration.xml
file, however it is easier to find the less detailed settings if they are all located at the top of the file. - Save and close the file.
Note
forceFailover
through the JMX Console or the Admin Console on the core server object.
JBOSS_DIST/jboss-as/server/<PROFILE>/deploy/hornetq/hornetq-jms.xml
.
Procedure 38.4. Configure HA Connection Factories
- Navigate to
$JBOSS_HOME/server/HornetQ_Colocated/deploy/hornetq/
- Open
hornetq-jms.xml
. - Add the following attributes and values as specified below.
- <ha>true</ha>
- Specifies the client must support high availability, and must always be true for fail over to occur.
- <retry-interval>1000</retry-interval>
- Specifies how long the client must wait (in milliseconds) before it can reconnect to the server.
- <retry-interval-multiplier>1.0</retry-interval-multiplier>
- Specifies the multiplier <retry-interval> used for each subsequent reconnection pauses. By setting the value to
1.0
, the retry interval is the same for each client reconnection request. - <reconnect-attempts>-1</reconnect-attempts>
- Specifies how many reconnect attempts a client should make before failing. Setting
-1
means unlimited reconnection attempts.
<?xml version='1.0' encoding='UTF-8'?> <configuration xmlns="urn:hornetq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd"> <connection-factory name="NettyConnectionFactory"> <xa>true</xa> <connectors> <connector-ref connector-name="netty"/> </connectors> <entries> <entry name="/ConnectionFactory"/> <entry name="/XAConnectionFactory"/> </entries> <ha>true</ha> <!-- Pause 1 second between connect attempts --> <retry-interval>1000</retry-interval> <!-- Multiply subsequent reconnect pauses by this multiplier. This can be used to implement an exponential back-off. For our purposes we just set to 1.0 so each reconnect pause is the same length --> <retry-interval-multiplier>1.0</retry-interval-multiplier> <!-- Try reconnecting an unlimited number of times (-1 means unlimited) --> <reconnect-attempts>-1</reconnect-attempts> </connection-factory> </configuration>
- Define new queues in both master and backup nodes by adding one of the following configuration blocks to the specified file.For
production/deploy/hornetq/hornetq-jms.xml
<queue name="testQueue"> <entry name="/queue/testQueue"/> <durable>true</durable> </queue>
Forproduction/deploy/customName-hornetq-jms.xml
Note
Ensure the file is well-formed from an XML validation perspective by ensuring the XML Namespace is present and correct in the file as specified.<configuration xmlns="urn:hornetq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd"> <queue name="testQueue"> <entry name="/queue/testQueue"/> <durable>true</durable> </queue> </configuration>
38.1.2. Colocated Backup Server
hornetq-jboss-beans.xml
and a hornetq-configuration.xml
configuration file. Any JMS components are created from the shared journal when the backup server becomes live (configured in Procedure 38.2, “Configure Shared Store and Journaling”).
Procedure 38.5. Create Backup Server
Important
- Navigate to
$JBOSS_HOME/server/HornetQ_Colocated/deploy/
- Create a new directory called
hornetq-backup1
. Move into that directory. - Open a text editor and create a new file called
hornetq-jboss-beans.xml
in thehornetq-backup1
directory. - Copy the following configuration into
hornetq-jboss-beans.xml
.<?xml version="1.0" encoding="UTF-8"?> <deployment xmlns="urn:jboss:bean-deployer:2.0"> <!-- The core configuration --> <bean name="BackupConfiguration" class="org.hornetq.core.config.impl.FileConfiguration"> <property name="configurationUrl">${jboss.server.home.url}/deploy/hornetq-backup1/hornetq-configuration.xml</property> </bean> <!-- The core server --> <bean name="BackupHornetQServer" class="org.hornetq.core.server.impl.HornetQServerImpl"> <constructor> <parameter> <inject bean="BackupConfiguration"/> </parameter> <parameter> <inject bean="MBeanServer"/> </parameter> <parameter> <inject bean="HornetQSecurityManager"/> </parameter> </constructor> <start ignored="true"/> <stop ignored="true"/> </bean> <!-- The JMS server --> <bean name="BackupJMSServerManager" class="org.hornetq.jms.server.impl.JMSServerManagerImpl"> <constructor> <parameter> <inject bean="BackupHornetQServer"/> </parameter> </constructor> </bean> </deployment>
- Save and close the file.
hornetq-jboss-beans.xml
file in Procedure 38.5, “Create Backup Server” contains configuration worth exploring in more detail. The BackupConfiguration bean is configured to pick up the configuration in hornetq-configuration.xml
. This file is created in the next procedure: Procedure 38.6, “Create Backup Server Configuration File”.
Note
Procedure 38.6. Create Backup Server Configuration File
- Navigate to
$JBOSS_HOME/server/HornetQ_Colocated/deploy/hornetq-backup1
- Open a text editor and create a new file called
hornetq-configuration.xml
in thehornetq-backup1
directory. - Copy the following configuration into
hornetq-configuration.xml
<?xml version="1.0" encoding="UTF-8"?> <configuration xmlns="urn:hornetq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd"> <jmx-domain>org.hornetq.backup1</jmx-domain> <clustered>true</clustered> <backup>true</backup> <shared-store>true</shared-store> <allow-failback>true</allow-failback> <file-deployment-enabled>true</file-deployment-enabled> <log-delegate-factory-class-name> org.hornetq.integration.logging.Log4jLogDelegateFactory </log-delegate-factory-class-name> <bindings-directory> /media/shared/data/hornetq-backup/bindings </bindings-directory> <journal-directory>/media/shared/data/hornetq-backup/journal</journal-directory> <journal-min-files>10</journal-min-files> <large-messages-directory> /media/shared/data/hornetq-backup/largemessages </large-messages-directory> <paging-directory>/media/shared/data/hornetq-backup/paging</paging-directory> <connectors> <connector name="netty-connector"> <factory-class> org.hornetq.core.remoting.impl.netty.NettyConnectorFactory </factory-class> <param key="host" value="${jboss.bind.address:localhost}"/> <param key="port" value="${hornetq.remoting.netty.backup.port:5446}"/> </connector> <connector name="in-vm"> <factory-class> org.hornetq.core.remoting.impl.invm.InVMConnectorFactory </factory-class> <param key="server-id" value="${hornetq.server-id:0}"/> </connector> </connectors> <acceptors> <acceptor name="netty"> <factory-class> org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory </factory-class> <param key="host" value="${jboss.bind.address:localhost}"/> <param key="port" value="${hornetq.remoting.netty.backup.port:5446}"/> </acceptor> </acceptors> <broadcast-groups> <broadcast-group name="bg-group1"> <group-address>231.7.7.7</group-address> <group-port>9876</group-port> <broadcast-period>1000</broadcast-period> <connector-ref>netty-connector</connector-ref> </broadcast-group> </broadcast-groups> <discovery-groups> <discovery-group name="dg-group1"> <group-address>231.7.7.7</group-address> <group-port>9876</group-port> <refresh-timeout>60000</refresh-timeout> </discovery-group> </discovery-groups> <cluster-connections> <cluster-connection name="my-cluster"> <address>jms</address> <connector-ref>netty-connector</connector-ref> <discovery-group-ref discovery-group-name="dg-group1"/> </cluster-connection> </cluster-connections> <security-settings> <security-setting match="#"> <permission type="createNonDurableQueue" roles="guest"/> <permission type="deleteNonDurableQueue" roles="guest"/> <permission type="consume" roles="guest"/> <permission type="send" roles="guest"/> </security-setting> </security-settings> <address-settings> <!--default for catch all--> <address-setting match="#"> <dead-letter-address>jms.queue.DLQ</dead-letter-address> <expiry-address>jms.queue.ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <max-size-bytes>10485760</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>BLOCK</address-full-policy> </address-setting> </address-settings> </configuration>
- Save and close the file.
hornetq-configuration.xml
file in Procedure 38.6, “Create Backup Server Configuration File” contains specific configuration which is discussed in hornetq-configuration.xml Configuration Points.
hornetq-configuration.xml Configuration Points
- <jmx-domain>org.hornetq.backup1</jmx-domain>
- Specifies the object name (in this case the backup server) in the Java Management Extensions (JMX) service. The default value is
org.hornetq
, however this name is already in use in other parts of HornetQ. You must change the name to a unique, system-wide name to avoid naming conflicts with the live server. - <clustered>true</clustered>
- Specifies whether the server should join a cluster. This configuration is the same as the live server.
- <backup>true</backup>
- Specifies whether the server starts as a backup server, and not a live server. Specifying true sets the server to start as a backup server.
- <shared-store>true</shared-store>
- Specifies whether the server should reference a shared store for journaling. This configuration is the same as the live server.
- <allow-failback>true</allow-failback>
- Specifies whether the backup server automatically stops and returns to standby mode when the live server becomes available again. If set to
false
, the server must be stopped manually to trigger a return to standby mode. - <bindings-directory>, <journal-directory>, <large-messages-directory>, <paging-directory>
- The paths in these elements must all resolve to the same paths the live server references. This ensures the backup server uses the same journaling files as the live server.
- <connectors>
- Two connectors are defined that allow clients to connect to the backup server once live: one connector for the netty connector factory (to allow client and server connections across different Virtual Machines); and one connector to allow the server to accept connections within the VM.
- <acceptors>
- The NettyAcceptorFactory is chosen here for VM compatibility.
- <broadcast-groups>, <discovery-groups>, <cluster-connections>, <security-settings>, <address-settings>
- The settings in these configuration blocks are standard settings.
Task: Create Configuration for Second Server Instance
Prerequisites
- Navigate to
.<JBOSS_HOME>
/server/ - Copy the
HornetQ_Colocated
directory, and rename it toHornetQ_Colocated_Second
. - Rename
<JBOSS_HOME>/server/HornetQ_Colocated_Second/hornetq-backup1/
to<JBOSS_HOME>/server/HornetQ_Colocated_Second/hornetq-backup-serverA/
- Open
<JBOSS_HOME>/server/HornetQ_Colocated_Second/hornetq/hornetq-configuration.xml
- For all parameters with data directories specified in
hornetq-configuration.xml
, change the data paths to/media/shared/data/hornetq-backup
.For example change:<bindings-directory> /media/shared/data/serverA/bindings </bindings-directory>to <bindings-directory> /media/shared/data/hornetq-backup/bindings </bindings-directory> - Open
<JBOSS_HOME>/server/HornetQ_Colocated_Second/hornetq-backup-serverA/hornetq-configuration.xml
- For all parameters with data directories specified in
hornetq-configuration.xml
, change the data paths to/media/shared/data/serverA
.For example change:<bindings-directory> /media/shared/data/hornetq-backup/bindings </bindings-directory>to <bindings-directory> /media/shared/data/serverA/bindings </bindings-directory>