Red Hat Summit이 콘텐츠는 선택한 언어로 제공되지 않습니다.
Chapter 6. Clustering
When clustering Red Hat JBoss BPM Suite, consider which components need to be clustered. You can cluster the following:
- GIT repository: virtual-file-system (VFS) repository that holds the business assets so that all cluster nodes use the same repository
- Execution Server and Web applications: the runtime server that resides in the container (such as, Red Hat JBoss EAP) along with BRMS and BPM Suite web applications so that nodes share the same runtime data.For instructions on clustering the application, refer to the container clustering documentation.
- Back-end database: database with the state data, such as, process instances, KIE sessions, history log, etc., for fail-over purposes
Figure 6.1. Schema of Red Hat JBoss BPM Suite system with individual system components
GIT Repository Clustering Mechanism
To cluster the GIT repository the following is used:
- Apache Zookeeper brings all parts together.
- Apache Helix is the cluster management component that registers all cluster details (the cluster itself, nodes, resources).
The runtime environment, that is the Execution Server, utilizes the following to provide the clustering capabilities:
- uberfire framework which provides the backbone of the web applications
Figure 6.2. Clustering schema with Helix and Zookeeper
A typical clustering setup involves the following:
- Setting up the cluster itself using Zookeeper and Helix
- Setting up the back-end database with Quartz tables and configuration
- Configuring clustering on your container (this documentation provides only clustering instructions for Red Hat JBoss EAP 6)
Clustering Maven Repositories
Various operations within the Business Central publish JARs to the Business Central's internal Maven Repository.
This repository exists on the application server's file-system as regular files and is not cluster aware. This folder is not synchronized across the various nodes in the cluster and must be synchronized using external tools like
rsync.
An alternate to the use of an external synchronization tool is to set the system property org.guvnor.m2repo.dir on each cluster node to point to a SAN or NAS. In this case clustering of the Maven repository folder is not needed.
6.1. Clustering on JBoss EAP
링크 복사링크가 클립보드에 복사되었습니다!
To install JBoss BPM Suite in clustered mode, we recommend you use the JAR installer, which provides a sample setup that works out of the box. You can, however, set up clustering with the deployable ZIP for EAP as well.
6.1.1. Clustering using JAR Installer
링크 복사링크가 클립보드에 복사되었습니다!
Note
The JAR installer provides sample setup only, adjusting the configuration is necessary for it to suit your project's needs.
Using the JAR installer described in Section 2.1, “The Red Hat JBoss BPM Suite Installer installation”, you can set up a basic clustering configuration of JBoss BPM Suite.
The automatic configuration creates three ZooKeeper instances, a Helix cluster that uses these instances, and two Quartz datastores (one managed and one unmanaged). This JBoss BPM Suite setup consists of two EAP nodes that share a Maven repository, use Quartz for coordinating timed tasks, and have
business-central.war, dashbuilder.war, and kie-server.war deployed. To customize the setup to fit your scenario, or to use clustering with the deployable ZIP, see the Section 6.1.2, “Custom Configuration (Deployable ZIP)”. You can also get more information in the JBoss EAP documentation.
Follow the installation process described in Section 2.1.1, “Installing Red Hat JBoss BPM Suite Using the Installer” and select Install clustered configuration in Advanced Runtime Configuration. After clicking next, you will go through the following steps:
Note
The steps listed here describe the GUI installation. The steps for the console installation are analogous.
Select JDBC provider
On this screen, select the JDBC provider from the list. You need to provide the corresponding JDBC driver JAR(s) in one of these ways:- Select one or more files on the filesystem
- Provide one or more URLs. The installer downloads the files automatically.
The installer then copies the JAR(s) into the appropriate location under the directory$EAP_HOME/modules, where a correspondingmodule.xmlfile is also created automatically.Figure 6.3. JDBC Driver setup
Configure Quartz connection
On the next screen, provide the data to the database for Quartz. The installer automatically creates the Quartz definition file ($EAP_HOME/domain/configuration/quartz-definition.properties) and two Quartz datasources in the domain configuration file$EAP_HOME/domain/domain.xml. You may edit the files after having finished the installation.Note
During the installation, Quartz DDL scripts will be run on the database selected in this step. These scripts make changes needed for Quartz to operate (adding tables, etc.), and can be found in$EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/ddl-scriptsfor reference (You do not need to modify them in any way).Figure 6.4. Quartz database setup
- Click next to initiate the installation.
Important
When using the JAR installer, thewararchives are automatically created from the applications residing in$EAP_HOME/standalone/deployments/. That means additional space is necessary as the applications exist both in uncompressed and compressed state in the storage during the installation.Three ZooKeeper instances are automatically created in$EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/(directory nameszookeeper-one,zookeeper-two, andzookeeper-three).In the directory$EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/helix-core, you can find the default Helix configuration and the scripts to launch the cluster—startCluster.shfor UNIX andstartCluster.batfor Windows.After the installation finishes, DO NOT select to run the server immediately. You first need to start the cluster by moving to the directory$EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/helix-coreand executing the aforementioned launch script:On UNIX systems:./startCluster.sh
./startCluster.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow On Windows:./startCluster.bat
./startCluster.batCopy to Clipboard Copied! Toggle word wrap Toggle overflow This script launches the Helix cluster and the ZooKeeper instances. Only after that, start the EAP server in domain mode by moving to the directory$EAP_HOME/binand running:On UNIX systems:./domain.sh
./domain.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow On Windows:./domain.bat
./domain.batCopy to Clipboard Copied! Toggle word wrap Toggle overflow
You now have a fully functioning JBoss BPM Suite cluster.
6.1.2. Custom Configuration (Deployable ZIP)
링크 복사링크가 클립보드에 복사되었습니다!
When using JBoss EAP clustering, a single JBoss EAP domain controller exists with other JBoss EAP slaves connecting to it as management users. Deployment of Business Central and dashbuilder can be done as a management user on a domain controller, and the WAR deployments will be distributed to other members of the JBoss EAP cluster.
To configure clustering on Red Hat JBoss EAP 6, do the following:
- Configure ZooKeeper and Helix according to Section 6.2.1, “Setting up a Cluster”.
- Configure Quartz according to Section 6.2.2, “Setting up Quartz”.
- Install your JDBC driver as a core module: copy the driver jar to
$EAP_HOME/modules/system/layers/base/and create amodule.xmlfile in the directory. - Edit the
module.xmlfile as of the respective module XSD.Example 6.1. The module.xml file content for a PostgreSQL datasource
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Configure the data source for the server: open for editing the
host.xmlorstandalone.xmlfile, depending on the used PROFILE located in$EAP_HOME/PROFILE/, locate thefullprofile, and do the following:- Add the definition of the main datasource used by JBoss BPM Suite.
Example 6.2. The PostgreSQL datasource defined as the main JBoss BPM Suite datasource
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Add the definition of the data source for the Quartz service.
Example 6.3. The PostgreSQL datasource defined as the Quartz datasource
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Define the data source driver.
Example 6.4. The PostgreSQL driver definition
<driver name="postgres" module="org.postgresql"> <xa-datasource-class>org.postgresql.xa.PGXADataSource</xa-datasource-class> </driver>
<driver name="postgres" module="org.postgresql"> <xa-datasource-class>org.postgresql.xa.PGXADataSource</xa-datasource-class> </driver>Copy to Clipboard Copied! Toggle word wrap Toggle overflow
- Configure individual server nodes in the
main-server-groupelement in the$EAP_HOME/domain/configuration/host.xmlfile with properties defined in Table 6.1, “Cluster node properties”:Note that a when configuring a JBoss EAP cluster with Zookeeper, a different number of JBoss EAP nodes than Zookeeper nodes is possible (keeping in mind that Zookeeper should to have an odd number of nodes). However, having the same node count for both Zookeeper and JBoss EAP is considered best practice.Expand Table 6.1. Cluster node properties Property name Value Description jboss.node.namenodeOnenode name unique within the clusterorg.quartz.properties/bpms/quartz-definition.propertiesabsolute path to the Quartz configuration fileorg.uberfire.cluster.idbpms-clusterHelix cluster nameorg.uberfire.cluster.local.idnodeOne_12345unique ID of the Helix cluster nodeNote that:is replaced with_.org.uberfire.cluster.vfs.lockvfs-reponame of the resource defined on the Helix clusterorg.uberfire.cluster.zkserver1:2181Zookeeper locationorg.uberfire.metadata.index.dir/home/jbpm/node[N]/indexlocation where the index for search is to be created (maintained by Apache Lucene)org.uberfire.nio.git.daemon.hostnodeOnethe name of the daemon host machine in a physical cluster. org.uberfire.nio.git.daemon.port9418port used by the VFS repo to accept client connectionsThe port must be unique for each cluster member.org.uberfire.nio.git.dir/home/jbpm/node[N]/repoGIT (VFS) repository location on node[N]org.uberfire.nio.git.ssh.hostnodeOnethe name of the SSH host machine in a physical cluster. org.uberfire.nio.git.ssh.port8003the unique port number for ssh access to the GIT repo for a cluster running on physical machines. org.uberfire.nio.git.ssh.hostport and org.uberfire.nio.git.daemon.hostport8003 and 9418In a virtualized environment, the outside port to be used. Example 6.5. Cluster nodeOne configuration
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example 6.6. Cluster nodeTwo configuration
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example 6.7. Cluster nodeThree configuration
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Add management users as instructed in the Administration and Configuration Guide for Red Hat JBoss EAP and application users as instructed in Red Hat JBoss BPM Suite Administration and Configuration Guide.
- Move to the directory
$EAP_HOME/binand start the application server in domain mode:On UNIX systems:./domain.sh
./domain.shCopy to Clipboard Copied! Toggle word wrap Toggle overflow On Windows:./domain.bat
./domain.batCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Check that the nodes are available.
Deploy the Business Central application to your servers:
- Change the predefined persistence of the application to the required data base (PostgreSQL): in
persistence.xmlchange the following:- jta-data-source name to the source defined on the application server (
java:jboss/datasources/psbpmsDS) - hibernate dialect to be match the data source dialect (
org.hibernate.dialect.PostgreSQLDialect)
- Log on as the management user to the server Administration console of your domain and add the new deployments using the Runtime view of the console. Once the deployment is added to the domain, assign it to the correct server group (
main-server-group).
Note
It is important users explicitly check deployment unit readiness with every cluster member.
When a deployment unit is created on a cluster node, it takes some time before it is distributed among all cluster members. Deployment status can be checked via UI and REST, however if the query goes to the node where the deployment was originally issued, the answer is
deployed. Any request targeting this deployment unit sent to a different cluster member fails with DeploymentNotFoundException.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}