Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 6. Clustering

When clustering Red Hat JBoss BPM Suite, consider which components need to be clustered. You can cluster the following:
  • GIT repository: virtual-file-system (VFS) repository that holds the business assets so that all cluster nodes use the same repository
  • Execution Server and Web applications: the runtime server that resides in the container (such as, Red Hat JBoss EAP) along with BRMS and BPM Suite web applications so that nodes share the same runtime data.
    For instructions on clustering the application, refer to the container clustering documentation.
  • Back-end database: database with the state data, such as, process instances, KIE sessions, history log, etc., for fail-over purposes
Schema of Red Hat JBoss BPM Suite system with individual system components

Figure 6.1. Schema of Red Hat JBoss BPM Suite system with individual system components

GIT Repository Clustering Mechanism

To cluster the GIT repository the following is used:
  • Apache Zookeeper brings all parts together.
  • Apache Helix is the cluster management component that registers all cluster details (the cluster itself, nodes, resources).
The runtime environment, that is the Execution Server, utilizes the following to provide the clustering capabilities:
  • uberfire framework which provides the backbone of the web applications
Description

Figure 6.2. Clustering schema with Helix and Zookeeper

A typical clustering setup involves the following:
  • Setting up the cluster itself using Zookeeper and Helix
  • Setting up the back-end database with Quartz tables and configuration
  • Configuring clustering on your container (this documentation provides only clustering instructions for Red Hat JBoss EAP 6)

Clustering Maven Repositories

Various operations within the Business Central publish JARs to the Business Central's internal Maven Repository.
This repository exists on the application server's file-system as regular files and is not cluster aware. This folder is not synchronized across the various nodes in the cluster and must be synchronized using external tools like rsync.
An alternate to the use of an external synchronization tool is to set the system property org.guvnor.m2repo.dir on each cluster node to point to a SAN or NAS. In this case clustering of the Maven repository folder is not needed.

6.1. Clustering on JBoss EAP
Copy link

To install JBoss BPM Suite in clustered mode, we recommend you use the JAR installer, which provides a sample setup that works out of the box. You can, however, set up clustering with the deployable ZIP for EAP as well.

6.1.1. Clustering using JAR Installer
Copy link

Note

The JAR installer provides sample setup only, adjusting the configuration is necessary for it to suit your project's needs.
Using the JAR installer described in Section 2.1, “The Red Hat JBoss BPM Suite Installer installation”, you can set up a basic clustering configuration of JBoss BPM Suite.
The automatic configuration creates three ZooKeeper instances, a Helix cluster that uses these instances, and two Quartz datastores (one managed and one unmanaged). This JBoss BPM Suite setup consists of two EAP nodes that share a Maven repository, use Quartz for coordinating timed tasks, and have business-central.war, dashbuilder.war, and kie-server.war deployed. To customize the setup to fit your scenario, or to use clustering with the deployable ZIP, see the Section 6.1.2, “Custom Configuration (Deployable ZIP)”. You can also get more information in the JBoss EAP documentation.
Follow the installation process described in Section 2.1.1, “Installing Red Hat JBoss BPM Suite Using the Installer” and select Install clustered configuration in Advanced Runtime Configuration. After clicking next, you will go through the following steps:

Note

The steps listed here describe the GUI installation. The steps for the console installation are analogous.

  1. Select JDBC provider

    On this screen, select the JDBC provider from the list. You need to provide the corresponding JDBC driver JAR(s) in one of these ways:
    • Select one or more files on the filesystem
    • Provide one or more URLs. The installer downloads the files automatically.
    The installer then copies the JAR(s) into the appropriate location under the directory $EAP_HOME/modules, where a corresponding module.xml file is also created automatically.
    Configure JDBC provider and drivers

    Figure 6.3. JDBC Driver setup

  2. Configure Quartz connection

    On the next screen, provide the data to the database for Quartz. The installer automatically creates the Quartz definition file ($EAP_HOME/domain/configuration/quartz-definition.properties) and two Quartz datasources in the domain configuration file $EAP_HOME/domain/domain.xml. You may edit the files after having finished the installation.

    Note

    During the installation, Quartz DDL scripts will be run on the database selected in this step. These scripts make changes needed for Quartz to operate (adding tables, etc.), and can be found in $EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/ddl-scripts for reference (You do not need to modify them in any way).
    Configure database connections

    Figure 6.4. Quartz database setup

  3. Click next to initiate the installation.

    Important

    When using the JAR installer, the war archives are automatically created from the applications residing in $EAP_HOME/standalone/deployments/. That means additional space is necessary as the applications exist both in uncompressed and compressed state in the storage during the installation.
    Three ZooKeeper instances are automatically created in $EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/ (directory names zookeeper-one, zookeeper-two, and zookeeper-three).
    In the directory $EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/helix-core, you can find the default Helix configuration and the scripts to launch the cluster—startCluster.sh for UNIX and startCluster.bat for Windows.
    After the installation finishes, DO NOT select to run the server immediately. You first need to start the cluster by moving to the directory $EAP_HOME/jboss-brms-bpmsuite-6.2-supplementary-tools/helix-core and executing the aforementioned launch script:
    On UNIX systems: 
    ./startCluster.sh
    On Windows: 
    ./startCluster.bat
    This script launches the Helix cluster and the ZooKeeper instances. Only after that, start the EAP server in domain mode by moving to the directory $EAP_HOME/bin and running:
    On UNIX systems: 
    ./domain.sh
    On Windows: 
    ./domain.bat
You now have a fully functioning JBoss BPM Suite cluster.

6.1.2. Custom Configuration (Deployable ZIP)
Copy link

When using JBoss EAP clustering, a single JBoss EAP domain controller exists with other JBoss EAP slaves connecting to it as management users. Deployment of Business Central and dashbuilder can be done as a management user on a domain controller, and the WAR deployments will be distributed to other members of the JBoss EAP cluster.
To configure clustering on Red Hat JBoss EAP 6, do the following:
  1. Configure ZooKeeper and Helix according to Section 6.2.1, “Setting up a Cluster”.
  2. Configure Quartz according to Section 6.2.2, “Setting up Quartz”.
  3. Install your JDBC driver as a core module: copy the driver jar to $EAP_HOME/modules/system/layers/base/ and create a module.xml file in the directory.
  4. Edit the module.xml file as of the respective module XSD.

    Example 6.1. The module.xml file content for a PostgreSQL datasource

    <module xmlns="urn:jboss:module:1.0" name="org.postgresql">
	  <resources>
		<resource-root path="postgresql-jdbc.jar"/>
	  </resources>

	  <dependencies>
		<module name="javax.api"/>
		<module name="javax.transaction.api"/>
	  </dependencies>
	</module>
  5. Configure the data source for the server: open for editing the host.xml or standalone.xml file, depending on the used PROFILE located in $EAP_HOME/PROFILE/, locate the full profile, and do the following:
    1. Add the definition of the main datasource used by JBoss BPM Suite.

      Example 6.2. The PostgreSQL datasource defined as the main JBoss BPM Suite datasource

      <datasource jndi-name="java:jboss/datasources/psbpmsDS"
					pool-name="postgresDS" enabled="true" use-java-context="true">
	  <connection-url>jdbc:postgresql://localhost:5432/jbpm</connection-url>
	  <driver>postgres</driver>
	  <security>
		<user-name>bpms</user-name>
		<password>bpms</password>
		</security>
	</datasource>
    2. Add the definition of the data source for the Quartz service.

      Example 6.3. The PostgreSQL datasource defined as the Quartz datasource

      <datasource jta="false" jndi-name="java:jboss/datasources/quartzNotManagedDS"
			   pool-name="quartzNotManagedDS" enabled="true" use-java-context="true">
		<connection-url>jdbc:postgresql://localhost:5432/jbpm</connection-url>
	  <driver>postgres</driver>
	  <security>
		<user-name>bpms</user-name>
		<password>bpms</password>
	  </security>
	</datasource>
    3. Define the data source driver.

      Example 6.4. The PostgreSQL driver definition

       <driver name="postgres" module="org.postgresql">
	  <xa-datasource-class>org.postgresql.xa.PGXADataSource</xa-datasource-class>
	</driver>
  6. Configure individual server nodes in the main-server-group element in the $EAP_HOME/domain/configuration/host.xml file with properties defined in Table 6.1, “Cluster node properties”:
    Note that a when configuring a JBoss EAP cluster with Zookeeper, a different number of JBoss EAP nodes than Zookeeper nodes is possible (keeping in mind that Zookeeper should to have an odd number of nodes). However, having the same node count for both Zookeeper and JBoss EAP is considered best practice.
    Expand
    Table 6.1. Cluster node properties
    Property name Value Description
    jboss.node.name nodeOne
    node name unique within the cluster
    org.quartz.properties /bpms/quartz-definition.properties
    absolute path to the Quartz configuration file
    org.uberfire.cluster.id bpms-cluster
    Helix cluster name
    org.uberfire.cluster.local.id nodeOne_12345
    unique ID of the Helix cluster node
    Note that : is replaced with _.
    org.uberfire.cluster.vfs.lock vfs-repo
    name of the resource defined on the Helix cluster
    org.uberfire.cluster.zk server1:2181
    Zookeeper location
    org.uberfire.metadata.index.dir /home/jbpm/node[N]/index
    location where the index for search is to be created (maintained by Apache Lucene)
    org.uberfire.nio.git.daemon.host nodeOne the name of the daemon host machine in a physical cluster.
    org.uberfire.nio.git.daemon.port 9418
    port used by the VFS repo to accept client connections
    The port must be unique for each cluster member.
    org.uberfire.nio.git.dir /home/jbpm/node[N]/repo
    GIT (VFS) repository location on node[N]
    org.uberfire.nio.git.ssh.host nodeOne the name of the SSH host machine in a physical cluster.
    org.uberfire.nio.git.ssh.port 8003 the unique port number for ssh access to the GIT repo for a cluster running on physical machines.
    org.uberfire.nio.git.ssh.hostport and org.uberfire.nio.git.daemon.hostport 8003 and 9418 In a virtualized environment, the outside port to be used.

    Example 6.5. Cluster nodeOne configuration

    <system-properties>

	  <property name="org.uberfire.nio.git.dir" value="/tmp/bpms/nodeone" boot-time="false"/>
	  <property name="jboss.node.name" value="nodeOne" boot-time="false"/>

	  <property name="org.uberfire.cluster.id" value="bpms-cluster" boot-time="false"/>
	  <property name="org.uberfire.cluster.zk" value="server1:2181,server2:2181,server3:2181" boot-time="false"/>
	  <property name="org.uberfire.cluster.local.id" value="nodeOne_12345" boot-time="false"/>
	  <property name="org.uberfire.cluster.vfs.lock" value="vfs-repo" boot-time="false"/>

	  <property name="org.uberfire.nio.git.daemon.host" value="nodeOne" />
	  <property name="org.uberfire.nio.git.daemon.port" value="9418" boot-time="false"/>
	  <property name="org.uberfire.nio.git.daemon.hostport" value="9418" boot-time="false"/>

	  <property name="org.uberfire.nio.git.ssh.port" value="8003" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.hostport" value="8003" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.host" value="nodeOne" />

	  <property name="org.uberfire.metadata.index.dir" value="/tmp/jbpm/nodeone" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.cert.dir" value="/tmp/jbpm/nodeone" boot-time="false"/>

	  <property name="org.quartz.properties" value="/tmp/jbpm/quartz/quartz-db-postgres.properties" boot-time="false"/>

	</system-properties>

    Example 6.6. Cluster nodeTwo configuration

    <system-properties>

	  <property name="org.uberfire.nio.git.dir" value="/tmp/bpms/nodetwo" boot-time="false"/>
	  <property name="jboss.node.name" value="nodeTwo" boot-time="false"/>

	  <property name="org.uberfire.cluster.id" value="bpms-cluster" boot-time="false"/>
	  <property name="org.uberfire.cluster.zk" value="server1:2181,server2:2181,server3:2181" boot-time="false"/>
	  <property name="org.uberfire.cluster.local.id" value="nodeTwo_12346" boot-time="false"/>
	  <property name="org.uberfire.cluster.vfs.lock" value="vfs-repo" boot-time="false"/>

	  <property name="org.uberfire.nio.git.daemon.host" value="nodeTwo" />
	  <property name="org.uberfire.nio.git.daemon.port" value="9418" boot-time="false"/>
	  <property name="org.uberfire.nio.git.daemon.hostport" value="9418" boot-time="false"/>

	  <property name="org.uberfire.nio.git.ssh.port" value="8003" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.hostport" value="8003" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.host" value="nodeTwo" />

	  <property name="org.uberfire.metadata.index.dir" value="/tmp/jbpm/nodetwo" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.cert.dir" value="/tmp/jbpm/nodetwo" boot-time="false"/>

	  <property name="org.quartz.properties" value="/tmp/jbpm/quartz/quartz-db-postgres.properties" boot-time="false"/>


	</system-properties>

    Example 6.7. Cluster nodeThree configuration

    <system-properties>

	  <property name="org.uberfire.nio.git.dir" value="/tmp/bpms/nodethree" boot-time="false"/>
	  <property name="jboss.node.name" value="nodeThree" boot-time="false"/>

	  <property name="org.uberfire.cluster.id" value="bpms-cluster" boot-time="false"/>
	  <property name="org.uberfire.cluster.zk" value="server1:2181,server2:2181,server3:2181" boot-time="false"/>
	  <property name="org.uberfire.cluster.local.id" value="nodeThree_12347" boot-time="false"/>
	  <property name="org.uberfire.cluster.vfs.lock" value="vfs-repo" boot-time="false"/>

	  <property name="org.uberfire.nio.git.daemon.host" value="nodeThree" />
	  <property name="org.uberfire.nio.git.daemon.port" value="9418" boot-time="false"/>
	  <property name="org.uberfire.nio.git.daemon.hostport" value="9418" boot-time="false"/>

	  <property name="org.uberfire.nio.git.ssh.port" value="8003" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.hostport" value="8003" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.host" value="nodeThree" />

	  <property name="org.uberfire.metadata.index.dir" value="/tmp/jbpm/nodethree" boot-time="false"/>
	  <property name="org.uberfire.nio.git.ssh.cert.dir" value="/tmp/jbpm/nodethree" boot-time="false"/>

	  <property name="org.quartz.properties" value="/tmp/jbpm/quartz/quartz-db-postgres.properties" boot-time="false"/>


	</system-properties>
  7. Add management users as instructed in the Administration and Configuration Guide for Red Hat JBoss EAP and application users as instructed in Red Hat JBoss BPM Suite Administration and Configuration Guide.
  8. Move to the directory $EAP_HOME/bin and start the application server in domain mode:
    On UNIX systems: 
    ./domain.sh
    On Windows: 
    ./domain.bat
  9. Check that the nodes are available.
Deploy the Business Central application to your servers:
  1. Change the predefined persistence of the application to the required data base (PostgreSQL): in persistence.xml change the following:
    1. jta-data-source name to the source defined on the application server (java:jboss/datasources/psbpmsDS)
    2. hibernate dialect to be match the data source dialect (org.hibernate.dialect.PostgreSQLDialect)
  2. Log on as the management user to the server Administration console of your domain and add the new deployments using the Runtime view of the console. Once the deployment is added to the domain, assign it to the correct server group (main-server-group).

Note

It is important users explicitly check deployment unit readiness with every cluster member.
When a deployment unit is created on a cluster node, it takes some time before it is distributed among all cluster members. Deployment status can be checked via UI and REST, however if the query goes to the node where the deployment was originally issued, the answer is deployed. Any request targeting this deployment unit sent to a different cluster member fails with DeploymentNotFoundException.
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top