Chapter 8. Clustered Enterprise JavaBeans
8.1. About Clustered Enterprise JavaBeans (EJBs)
EJB components can be clustered for high-availability scenarios. They use different protocols than HTTP components, so they are clustered in different ways. EJB 2 and 3 stateful and stateless beans can be clustered.
For information on singletons, see HA Singleton Service in the JBoss EAP Development Guide
8.2. EJB Client Code Simplification
You can simplify the EJB client code when invoking the EJB server-side clustered components. The following procedures outline the multiple ways to simplify the EJB client code:
The use of the jboss-ejb-client.properties
file is deprecated in favor of the wildfly-config.xml
file.
8.3. Deploying Clustered EJBs
Clustering support is available in the HA profiles of JBoss EAP 7.1. Starting the standalone server with HA capabilities enabled involves starting it with the standalone-ha.xml
or standalone-full-ha.xml
file:
$ EAP_HOME/bin/standalone.sh --server-config=standalone-ha.xml
This will start a single instance of the server with HA capabilities.
To be able to see the benefits of clustering, you will need more than one instance of the server. So let us start another server with HA capabilities. That another instance of the server can either be on the same machine or on some other machine. If it is on the same machine, you will need to take care of two things:
- Pass the port offset for the second instance
-
Make sure that each of the server instances have a unique
jboss.node.name
system property.
You can do that by passing the following two system properties to the startup command:
$ EAP_HOME/bin/standalone.sh --server-config=standalone-ha.xml -Djboss.socket.binding.port-offset=PORT_OFFSET -Djboss.node.name=UNIQUE_NODE_NAME
Follow whichever approach you feel comfortable with for deploying the EJB deployment to this instance too.
Deploying the application on just one node of a standalone instance of a clustered server does not mean that it will be automatically deployed to the other clustered instance. You will have to do deploy it explicitly on the other standalone clustered instance too. Or you can start the servers in domain mode so that the deployment can be deployed to all the servers within a server group.
Now that you have deployed an application with clustered EJBs on both the instances, the EJBs are now capable of making use of the clustering features.
Starting JBoss EAP 7, if JBoss EAP is started using an HA profile, the state of your SFSBs will be replicated. You no longer need to use the @Clustered
annotation to enable clustering behavior.
Disabling this behavior is achievable on a per-EJB basis by annotating your bean using @Stateful(passivationCapable=false)
, which is new to the EJB 3.2 specification; or globally, via the ejb3
subsystem.
8.4. Failover for Clustered EJBs
Clustered EJBs have failover capability. The state of the @Stateful
EJBs is replicated across the cluster nodes so that if one of the nodes in the cluster goes down, some other node will be able to take over the invocations.
Under some circumstances in a clustered environment, such as when a server in the cluster crashes, the EJB client might receive an exception instead of a response. The EJB client library will automatically retry the invocation when it is safe to do so, depending on the type of the failure that occurs. However, if a request fails and it cannot be determined conclusively to be safe to retry, then you can handle the exception as appropriate for your environment. You can, however, use custom interceptors to add additional retry behavior.
8.5. Remote Standalone Clients
The use of the jboss-ejb-client.properties
file is deprecated in favor of the wildfly-config.xml
file.
A standalone remote client can use either the JNDI approach or native JBoss EJB client APIs to communicate with the servers. The important thing to note is that when you are invoking clustered EJB deployments, you do not have to list all the servers within the cluster. This would not have been feasible due the dynamic nature of cluster node additions within a cluster.
The remote client has to list only one of the servers with the clustering capability. This server will act as the starting point for cluster topology communication between the client and the clustered nodes.
Note that you have to configure the ejb
cluster in the jboss-ejb-client.properties
configuration file:
remote.clusters=ejb remote.cluster.ejb.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false remote.cluster.ejb.connect.options.org.xnio.Options.SSL_ENABLED=false
8.6. Cluster Topology Communication
The use of the jboss-ejb-client.properties
file is deprecated in favor of the wildfly-config.xml
file.
When a client connects to a server, the JBoss EJB client implementation communicates internally with the server for the cluster topology information, if the server has clustering capability. For example, assuming that server X is listed as the initial server to connect to, when the client connects to server X, the server will send back an asynchronous cluster topology message to the client. This topology message consists of the cluster name and the information of the nodes that belong to the cluster. The node information includes the node address and port number to connect to, when required. So in this example, server X will send back the cluster topology consisting of the other server Y that belongs to the cluster.
In case of stateful clustered EJBs, the invocation flow happens in two steps.
- Creation of a session for the stateful bean, which happens when you do a JNDI lookup for that bean.
- Invocation of the returned proxy.
The lookup for the stateful bean, internally, triggers a synchronous session creation request from the client to the server. In this case, the session creation request goes to server X because it was configured in the jboss-ejb-client.properties
file. Since server X is clustered, it will return a session id and send back an affinity of that session. In case of clustered servers, the affinity is equal to the name of the cluster to which the stateful bean belongs on the server side. For non-clustered beans, the affinity is the node name on which the session was created. This affinity will help the EJB client to route the invocations on the proxy, as appropriate, to either a node within a cluster for clustered beans, or to a specific node for non-clustered beans. While this session creation request is going on, server X will also send back an asynchronous message that contains the cluster topology. The JBoss EJB client implementation will record this topology information and use it later for connection creation to nodes within the cluster and routing invocations to those nodes, when required.
To understand how failover works, consider the same example of server X being the starting point and a client application looking up a stateful bean and invoking it. During these invocations, the client side collects the cluster topology information from the server. Assuming that for some reason server X goes down and the client application subsequently invokes on the proxy. The JBoss EJB client implementation at this stage must be aware of the affinity, and in this case it is the cluster affinity. From the cluster topology information that the client has, it knows that the cluster has two nodes, server X and server Y. When the invocation arrives, the client notices that server X is down, so it uses a selector to fetch a suitable node from the cluster nodes. When the selector returns a node from the cluster nodes, the JBoss EJB client implementation creates a connection to that node, if the connection was not already created earlier, and creates an EJB receiver out of it. Since in this example, the only other node in the cluster is server Y, the selector will return server Y as the node and the JBoss EJB client implementation will use it to create an EJB receiver out of it and use this receiver to pass on the invocation on the proxy. Effectively, the invocation has now failed over to a different node within the cluster.
8.7. Automatic Transaction Stickiness for EJBs
A transaction object, which is looked up from the same context as the EJB proxy, targets the same host. Having an active transaction pins the invocation context to the same node, if the context is multi-host or clustered.
This behavior depends on whether you have outflowed your transaction or you are using a remote user transaction.
For an outflowed transaction, when an application is looked up on a specific node, all the invocations to that application under the same transaction attempt to target this node. The nodes that have already received the outflowed transaction will be preferred over nodes that have not received it yet.
For a remote user transaction, the first successful invocation will lock the transaction to the given node, and subsequent invocations under this transaction must go to the same node, otherwise an exception is thrown.
8.8. Remote Clients on Another Instance
This section explains how a client application deployed on a JBoss EAP instance invokes a clustered stateful bean that is deployed on another JBoss EAP instance.
In the following example, there are three servers involved. Servers X and Y both belong to a cluster and have clustered EJBs deployed on them. There is another server instance server C, which may or may not have clustering capability. Server C acts as a client on which there is a deployment that wants to invoke the clustered beans deployed on servers X and Y and achieve failover.
The configurations are done in the jboss-ejb-client.xml
file, which points to a remote outbound connection to the other server. The configuration in the jboss-ejb-client.xml
file is in the deployment of server C because server C is the client. The client configuration need not point to all the clustered nodes, but just to one of them. This will act as a starting point for the communication.
In this case, a remote outbound connection is created from server C to server X and then server X is used as the starting point for the communication. Similar to the case of remote standalone clients, when the application on server C looks up a stateful bean, a session creation request is sent to server X that returns a session id and the cluster affinity for it. Server X also sends back an asynchronous message to server C containing the cluster topology. This topology information includes the node information of server Y, because server Y belongs to the cluster along with server X. Subsequent invocations on the proxy will be routed appropriately to the nodes in the cluster. If server X goes down, as explained earlier, a different node from the cluster will be selected and the invocation will be forwarded to that node.
Both remote standalone clients as well as remote clients on another JBoss EAP instance act similarly in terms of failover.
8.9. Standalone and In-server Client Configuration
The use of the jboss-ejb-client.properties
file is deprecated in favor of the wildfly-config.xml
file.
To connect an EJB client to a clustered EJB application, you need to expand the existing configuration in standalone EJB client or in-server EJB client to include cluster connection configuration. The jboss-ejb-client.properties
for standalone EJB client, or even jboss-ejb-client.xml
file for a server-side application must be expanded to include a cluster configuration.
An EJB client is any program that uses an EJB on a remote server. A client is in-server
when the EJB client calling the remote server is itself running inside of a server. In other words, a JBoss EAP instance calling out to another JBoss EAP instance would be considered an in-server client.
This example shows the additional cluster configuration required for a standalone EJB client.
remote.clusters=ejb remote.cluster.ejb.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false remote.cluster.ejb.connect.options.org.xnio.Options.SSL_ENABLED=false remote.cluster.ejb.username=test remote.cluster.ejb.password=password
If an application uses the remote-outbound-connection, you need to configure the jboss-ejb-client.xml
file and add cluster configuration as shown in the following example:
<jboss-ejb-client xmlns:xsi="urn:jboss:ejb-client:1.2" xsi:noNamespaceSchemaLocation="jboss-ejb-client_1_2.xsd"> <client-context> <ejb-receivers> <!-- this is the connection to access the app-one --> <remoting-ejb-receiver outbound-connection-ref="remote-ejb-connection-1" /> <!-- this is the connection to access the app-two --> <remoting-ejb-receiver outbound-connection-ref="remote-ejb-connection-2" /> </ejb-receivers> <!-- If an outbound connection connects to a cluster, a list of members is provided after successful connection. To connect to this node this cluster element must be defined. --> <clusters> <!-- cluster of remote-ejb-connection-1 --> <cluster name="ejb" security-realm="ejb-security-realm-1" username="quickuser1"> <connection-creation-options> <property name="org.xnio.Options.SSL_ENABLED" value="false" /> <property name="org.xnio.Options.SASL_POLICY_NOANONYMOUS" value="false" /> </connection-creation-options> </cluster> </clusters> </client-context> </jboss-ejb-client>
For more information about remote-outbound-connection, see About the Remoting Subsystem in the JBoss EAP Configuration Guide.
For a secure connection you need to add the credentials to cluster configuration in order to avoid an authentication exception.
8.10. Implementing a Custom Load Balancing Policy for EJB Calls
The use of the jboss-ejb-client.properties
file is deprecated in favor of the wildfly-config.xml
file.
It is possible to implement an alternate or customized load balancing policy in order to balance an application’s EJB calls across servers.
You can implement AllClusterNodeSelector
for EJB calls. The node selection behavior of AllClusterNodeSelector
is similar to default selector except that AllClusterNodeSelector
uses all available cluster nodes even in case of a large cluster (number of nodes > 20). If an unconnected cluster node is returned, it is opened automatically. The following example shows AllClusterNodeSelector
implementation:
package org.jboss.as.quickstarts.ejb.clients.selector; import java.util.Arrays; import java.util.Random; import java.util.logging.Level; import java.util.logging.Logger; import org.jboss.ejb.client.ClusterNodeSelector; public class AllClusterNodeSelector implements ClusterNodeSelector { private static final Logger LOGGER = Logger.getLogger(AllClusterNodeSelector.class.getName()); @Override public String selectNode(final String clusterName, final String[] connectedNodes, final String[] availableNodes) { if(LOGGER.isLoggable(Level.FINER)) { LOGGER.finer("INSTANCE "+this+ " : cluster:"+clusterName+" connected:"+Arrays.deepToString(connectedNodes)+" available:"+Arrays.deepToString(availableNodes)); } if (availableNodes.length == 1) { return availableNodes[0]; } final Random random = new Random(); final int randomSelection = random.nextInt(availableNodes.length); return availableNodes[randomSelection]; } }
You can also implement the SimpleLoadFactorNodeSelector
for EJB calls. Load balancing in SimpleLoadFactorNodeSelector
happens based on a load factor. The load factor (2/3/4) is calculated based on the names of nodes (A/B/C) irrespective of the load on each node. The following example shows SimpleLoadFactorNodeSelector
implementation:
package org.jboss.as.quickstarts.ejb.clients.selector; import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.logging.Level; import java.util.logging.Logger; import org.jboss.ejb.client.DeploymentNodeSelector; public class SimpleLoadFactorNodeSelector implements DeploymentNodeSelector { private static final Logger LOGGER = Logger.getLogger(SimpleLoadFactorNodeSelector.class.getName()); private final Map<String, List<String>[]> nodes = new HashMap<String, List<String>[]>(); private final Map<String, Integer> cursor = new HashMap<String, Integer>(); private ArrayList<String> calculateNodes(Collection<String> eligibleNodes) { ArrayList<String> nodeList = new ArrayList<String>(); for (String string : eligibleNodes) { if(string.contains("A") || string.contains("2")) { nodeList.add(string); nodeList.add(string); } else if(string.contains("B") || string.contains("3")) { nodeList.add(string); nodeList.add(string); nodeList.add(string); } else if(string.contains("C") || string.contains("4")) { nodeList.add(string); nodeList.add(string); nodeList.add(string); nodeList.add(string); } } return nodeList; } @SuppressWarnings("unchecked") private void checkNodeNames(String[] eligibleNodes, String key) { if(!nodes.containsKey(key) || nodes.get(key)[0].size() != eligibleNodes.length || !nodes.get(key)[0].containsAll(Arrays.asList(eligibleNodes))) { // must be synchronized as the client might call it concurrent synchronized (nodes) { if(!nodes.containsKey(key) || nodes.get(key)[0].size() != eligibleNodes.length || !nodes.get(key)[0].containsAll(Arrays.asList(eligibleNodes))) { ArrayList<String> nodeList = new ArrayList<String>(); nodeList.addAll(Arrays.asList(eligibleNodes)); nodes.put(key, new List[] { nodeList, calculateNodes(nodeList) }); } } } } private synchronized String nextNode(String key) { Integer c = cursor.get(key); List<String> nodeList = nodes.get(key)[1]; if(c == null || c >= nodeList.size()) { c = Integer.valueOf(0); } String node = nodeList.get(c); cursor.put(key, Integer.valueOf(c + 1)); return node; } @Override public String selectNode(String[] eligibleNodes, String appName, String moduleName, String distinctName) { if (LOGGER.isLoggable(Level.FINER)) { LOGGER.finer("INSTANCE " + this + " : nodes:" + Arrays.deepToString(eligibleNodes) + " appName:" + appName + " moduleName:" + moduleName + " distinctName:" + distinctName); } // if there is only one there is no sense to choice if (eligibleNodes.length == 1) { return eligibleNodes[0]; } final String key = appName + "|" + moduleName + "|" + distinctName; checkNodeNames(eligibleNodes, key); return nextNode(key); } }
Configuring the jboss-ejb-client.properties File
You need to add the property remote.cluster.ejb.clusternode.selector
with the name of your implementation class (AllClusterNodeSelector
or SimpleLoadFactorNodeSelector
). The selector will see all configured servers that are available at the invocation time. The following example uses AllClusterNodeSelector
as the cluster node selector:
remote.clusters=ejb remote.cluster.ejb.clusternode.selector=org.jboss.as.quickstarts.ejb.clients.selector.AllClusterNodeSelector remote.cluster.ejb.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false remote.cluster.ejb.connect.options.org.xnio.Options.SSL_ENABLED=false remote.cluster.ejb.username=test remote.cluster.ejb.password=password remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED=false remote.connections=one,two remote.connection.one.host=localhost remote.connection.one.port = 8080 remote.connection.one.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false remote.connection.one.username=user remote.connection.one.password=user123 remote.connection.two.host=localhost remote.connection.two.port = 8180 remote.connection.two.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
Using EJB Client API
You need to add the property remote.cluster.ejb.clusternode.selector
to the list for the PropertiesBasedEJBClientConfiguration
constructor. The following example uses AllClusterNodeSelector
as the cluster node selector:
Properties p = new Properties(); p.put("remote.clusters", "ejb"); p.put("remote.cluster.ejb.clusternode.selector", "org.jboss.as.quickstarts.ejb.clients.selector.AllClusterNodeSelector"); p.put("remote.cluster.ejb.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false"); p.put("remote.cluster.ejb.connect.options.org.xnio.Options.SSL_ENABLED", "false"); p.put("remote.cluster.ejb.username", "test"); p.put("remote.cluster.ejb.password", "password"); p.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "false"); p.put("remote.connections", "one,two"); p.put("remote.connection.one.port", "8080"); p.put("remote.connection.one.host", "localhost"); p.put("remote.connection.two.port", "8180"); p.put("remote.connection.two.host", "localhost"); EJBClientConfiguration cc = new PropertiesBasedEJBClientConfiguration(p); ContextSelector<EJBClientContext> selector = new ConfigBasedEJBClientContextSelector(cc); EJBClientContext.setSelector(selector); p = new Properties(); p.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming"); InitialContext context = new InitialContext(p);
Configuring the jboss-ejb-client.xml File
To use the load balancing policy for server to server communication, package the class together with the application and configure it within the jboss-ejb-client.xml
settings located in META-INF
folder. The following example uses AllClusterNodeSelector
as the cluster node selector:
<jboss-ejb-client xmlns:xsi="urn:jboss:ejb-client:1.2" xsi:noNamespaceSchemaLocation="jboss-ejb-client_1_2.xsd"> <client-context deployment-node-selector="org.jboss.ejb.client.DeploymentNodeSelector"> <ejb-receivers> <!-- This is the connection to access the application. --> <remoting-ejb-receiver outbound-connection-ref="remote-ejb-connection-1" /> </ejb-receivers> <!-- Specify the cluster configurations applicable for this client context --> <clusters> <!-- Configure the cluster of remote-ejb-connection-1. --> <cluster name="ejb" security-realm="ejb-security-realm-1" username="test" cluster-node-selector="org.jboss.as.quickstarts.ejb.clients.selector.AllClusterNodeSelector"> <connection-creation-options> <property name="org.xnio.Options.SSL_ENABLED" value="false" /> <property name="org.xnio.Options.SASL_POLICY_NOANONYMOUS" value="false" /> </connection-creation-options> </cluster> </clusters> </client-context> </jboss-ejb-client>
To use the above configuration with security, you will need to add ejb-security-realm-1
to client-server configuration. The following example shows the CLI commands for adding security realm (ejb-security-realm-1
) the value is the base64 encoded password for the user "test":
/core-service=management/security-realm=ejb-security-realm-1:add() /core-service=management/security-realm=ejb-security-realm-1/server-identity=secret:add(value=cXVpY2sxMjMr)
If the load balancing policy should be used for server to server communication, the class can be packaged together with the application or as a module. This class is configured in the jboss-ejb-client
settings file located in the META-INF
directory of the top-level EAR archive. The following example uses RoundRobinNodeSelector
as the deployment node selector.
<jboss-ejb-client xmlns="urn:jboss:ejb-client:1.2"> <client-context deployment-node-selector="org.jboss.example.RoundRobinNodeSelector"> <ejb-receivers> <remoting-ejb-receiver outbound-connection-ref="..."/> </ejb-receivers> ... </client-context> </jboss-ejb-client>
If you are running a standalone server, use the start option -Djboss.node.name=
or the server configuration file standalone.xml
to configure the server name. Ensure that the server name is unique. If you are running a managed domain, the host controller automatically validates that the names are unique.
8.11. EJB Transactions in a Clustered Environment
If the client code invokes a clustered EJB, then the cluster affinity is set automatically. If you manage transactions on the client side, you can choose to target a specific node in the cluster or you can allow the client to lazily select the cluster node to handle transactions. This section describes both options.
EJB Transactions Target a Specific Node
You can target a specific node in the cluster to handle a transaction using the following procedure.
Specify the target cluster node address using the
PROVIDER_URL
property when creating theInitialContext
.props.put(Context.PROVIDER_URL, "remote+http://127.0.0.1:8080"); ... InitialContext ctx = new InitialContext(props);
In the client, look up the
txn:RemoteUserTransaction
from theInitialContext
.UserTransaction ut = (UserTransaction)ctx.lookup("txn:RemoteUserTransaction");
You can do a JNDI lookup for a
UserTransaction
by setting thePROVIDER_URL
property to the URL of the server and then look uptxn:UserTransaction
, as shown in the code example below:final Hashtable<String, String> jndiProperties = new Hashtable<>(); jndiProperties.put(Context.INITIAL_CONTEXT_FACTORY, "org.wildfly.naming.client.WildFlyInitialContextFactory"); jndiProperties.put(Context.PROVIDER_URL, "remote+http://localhost:8080"); final Context context = new InitialContext(jndiProperties); SecuredEJBRemote reference = (SecuredEJBRemote) context.lookup("txn:UserTransaction");
UserTransaction
is not bound to any particular destination until an actual invocation takes place. Upon invocation, thisUserTransaction
is bound to the respective destination for the entire lifetime of the transaction.You do not need to know the node name or the destination before beginning a
UserTransaction
. Theorg.jboss.ejb.client.EJBClient.getUserTransaction()
method gives you a remoteUserTransaction
that automatically selects its destination based on the first invocation. Looking up a remoteUserTransaction
from JNDI also works the same way.NoteThe
org.jboss.ejb.client.EJBClient.getUserTransaction()
method is deprecated.- When the transaction begins, all EJB invocations are then bound to that specific node for duration of the transaction, establishing server affinity.
- When the transaction ends, the server affinity is released, and the EJB proxies return to a general cluster affinity.
EJB Transactions Lazily Select a Node
You can allow the client to lazily select the cluster node to handle transactions during the first invocation pertaining to a transaction. This allows for load balancing of transactions across the cluster. To use this option, follow the procedure below.
-
Do not specify the
PROVIDER_URL
property in theInitialContext
used to invoke the EJBs. In the client, look up the
txn:RemoteUserTransaction
from theInitialContext
.UserTransaction ut = (UserTransaction)ctx.lookup("txn:RemoteUserTransaction");
- When the transaction begins, one cluster node is selected automatically, establishing server affinity, and all EJB invocations are then bound to that specific node for duration of the transaction.
- When the transaction ends, the server affinity is released, and the EJB proxies return to a general cluster affinity.