Chapter 7. Advanced Tutorials
7.1. Example Workflow: Automated Transaction Recovery Feature When Scaling Down a Cluster Copy linkLink copied to clipboard!
This feature is provided as Technology Preview only. It is not supported for use in a production environment, and it might be subject to significant future changes. See Technology Preview Features Support Scope on the Red Hat Customer Portal for information about the support scope for Technology Preview features.
This tutorial is applicable only for OpenShift 3 and for recovering transactions without the use of the EAP operator.
This tutorial demonstrates the automated transaction recovery feature of the JBoss EAP for OpenShift image when scaling down a cluster. This tutorial uses the jta-crash-rec-eap quickstart example and the eap73-tx-recovery-s2i application template to show how XA transactions issued on the OpenShift pod, when terminated within the cluster’s scale down, are recovered by the dedicated migration pod. The equivalent application template for the JDK 11 image is eap73-openjdk11-tx-recovery-s2i.
This tutorial uses the amq-broker-72-openshift:1.1 imagestream to provide a message broker that is JMS-compliant. After you have set up the initial broker pod, you can quickly deploy the quickstart by using OpenShift Container Platform features.
As a prerequisite to using the quickstarts, you must create an imagestream from the AMQ Long Term Support (LTS) image and name the imagestream amq-broker-72-openshift:1.1. You can access the LTS Image by downloading the amq7/amq-broker-lts-rhel7:7.4 container image from the Red Hat Ecosystem Catalog. For more information about installing the AMQ Broker, see Deploying a basic broker.
The jta-crash-rec-eap7 quickstart uses the H2 database that is included with JBoss EAP. It is a lightweight, relational example datasource that is used for examples only. It is not robust or scalable, is not supported, and should not be used in a production environment.
Additional resources
- See Deploying a basic broker in the Deploying AMQ Broker on OpenShift Container Platform guide for information about creating a basic AMQ Broker.
7.1.1. Prepare for Deployment Copy linkLink copied to clipboard!
-
Log in to your OpenShift instance using the
oc logincommand. Create a new project.
oc new-project eap-tx-demo
$ oc new-project eap-tx-demoCopy to Clipboard Copied! Toggle word wrap Toggle overflow Add the view role to the
defaultservice account, which will be used to run the underlying pods. This enables the service account to view all the resources in theeap-tx-demonamespace, which is necessary for managing the cluster.oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default
$ oc policy add-role-to-user view system:serviceaccount:$(oc project -q):defaultCopy to Clipboard Copied! Toggle word wrap Toggle overflow For automated transaction recovery to work, the JBoss EAP application must use a
ReadWriteManypersistent volume.Provision the persistent volume expected by the
eap73-tx-recovery-s2iapplication template to hold the data for the${APPLICATION_NAME}-eap-claimpersistent volume claim.This example uses a persistent volume object provisioned using the NFS method with the following definition:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Update the
pathandserverfields in the above definition for your environment, and provision the persistent volume with the following command:oc create -f txpv.yaml persistentvolume "txpv" created
$ oc create -f txpv.yaml persistentvolume "txpv" createdCopy to Clipboard Copied! Toggle word wrap Toggle overflow oc get pv NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE txpv 1Gi RWX Retain Available 26s
$ oc get pv NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE txpv 1Gi RWX Retain Available 26sCopy to Clipboard Copied! Toggle word wrap Toggle overflow ImportantWhen using the NFS method to provision persistent volume objects for the
eap73-tx-recovery-s2iapplication template, ensure the mount point is exported with sufficient permissions. On the host from which the mount point is exported, perform the following:chmod -R 777 /mnt/mountpoint
# chmod -R 777 /mnt/mountpointCopy to Clipboard Copied! Toggle word wrap Toggle overflow cat /etc/exports /mnt/mountpoint *(rw,sync,anonuid=185,anongid=185)
# cat /etc/exports /mnt/mountpoint *(rw,sync,anonuid=185,anongid=185)Copy to Clipboard Copied! Toggle word wrap Toggle overflow exportfs -va exporting *:/mnt/mountpoint
# exportfs -va exporting *:/mnt/mountpointCopy to Clipboard Copied! Toggle word wrap Toggle overflow setsebool -P virt_use_nfs 1
# setsebool -P virt_use_nfs 1Copy to Clipboard Copied! Toggle word wrap Toggle overflow Replace
/mnt/mountpointpath above as appropriate for your environment.
7.1.2. Deployment Copy linkLink copied to clipboard!
Deploy the
jta-crash-rec-eap7quickstart using theeap73-tx-recovery-s2iapplication template. Specify the following:Example:
eap73-tx-recovery-s2iapplication template (JDK 8)oc new-app --template=eap73-tx-recovery-s2i \ --name=eap-app
$ oc new-app --template=eap73-tx-recovery-s2i \ --name=eap-appCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example:
eap73-openjdk11-tx-recovery-s2iapplication template (JDK 11)oc new-app --template=eap73-openjdk11-tx-recovery-s2i \ --name=eap-app
$ oc new-app --template=eap73-openjdk11-tx-recovery-s2i \ --name=eap-appCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Wait for the build to finish. You can see the status of the build using the
oc logs -f bc/eap-appcommand. Modify the
eap-appdeployment configuration with the definition ofJAVA_OPTS_APPENDandJBOSS_MODULES_SYSTEM_PKGS_APPENDenvironment variables.oc get dc NAME REVISION DESIRED CURRENT TRIGGERED BY eap-app 1 1 1 config,image(eap-app:latest) eap-app-amq 1 1 1 config,image(amq-broker-72-openshift:1.1) eap-app-migration 1 1 1 config,image(eap-app:latest)
$ oc get dc NAME REVISION DESIRED CURRENT TRIGGERED BY eap-app 1 1 1 config,image(eap-app:latest) eap-app-amq 1 1 1 config,image(amq-broker-72-openshift:1.1) eap-app-migration 1 1 1 config,image(eap-app:latest)Copy to Clipboard Copied! Toggle word wrap Toggle overflow oc set env dc/eap-app \ -e JBOSS_MODULES_SYSTEM_PKGS_APPEND="org.jboss.byteman" \ -e JAVA_OPTS_APPEND="-javaagent:/tmp/src/extensions/byteman/byteman.jar=script:/tmp/src/src/main/scripts/xa.btm" deploymentconfig "eap-app" updated
$ oc set env dc/eap-app \ -e JBOSS_MODULES_SYSTEM_PKGS_APPEND="org.jboss.byteman" \ -e JAVA_OPTS_APPEND="-javaagent:/tmp/src/extensions/byteman/byteman.jar=script:/tmp/src/src/main/scripts/xa.btm" deploymentconfig "eap-app" updatedCopy to Clipboard Copied! Toggle word wrap Toggle overflow This setting will notify the Byteman tracing and monitoring tool to modify the XA transactions processing in the following way:
- The first transaction is always allowed to succeed.
- When an XA resource executes phase 2 of the second transaction, the JVM process of the particular pod is halted.
7.1.3. Using the JTA Crash Recovery Application Copy linkLink copied to clipboard!
List running pods in the current namespace:
oc get pods | grep Running NAME READY STATUS RESTARTS AGE eap-app-2-r00gm 1/1 Running 0 1m eap-app-amq-1-mws7x 1/1 Running 0 2m eap-app-migration-1-lvfdt 1/1 Running 0 2m
$ oc get pods | grep Running NAME READY STATUS RESTARTS AGE eap-app-2-r00gm 1/1 Running 0 1m eap-app-amq-1-mws7x 1/1 Running 0 2m eap-app-migration-1-lvfdt 1/1 Running 0 2mCopy to Clipboard Copied! Toggle word wrap Toggle overflow Issue a new XA transaction.
- Launch the application by opening a browser and navigating to http://eap-app-eap-tx-demo.openshift.example.com/jboss-jta-crash-rec.
-
Enter
Mercedesinto the Key field, andBenzinto the Value field. Click the Submit button. - Wait for a moment, then click the Refresh Table link.
Notice how the table row containing the
Mercedesentry is updated withupdated via JMS.If it has not yet updated, click the Refresh Table link couple of times. Alternatively, you can inspect the log of theeap-app-2-r00gmpod to verify the transaction was handled properly:oc logs eap-app-2-r00gm | grep 'updated' INFO [org.jboss.as.quickstarts.xa.DbUpdaterMDB] (Thread-0 (ActiveMQ-client-global-threads-1566836606)) JTA Crash Record Quickstart: key value pair updated via JMS.
$ oc logs eap-app-2-r00gm | grep 'updated' INFO [org.jboss.as.quickstarts.xa.DbUpdaterMDB] (Thread-0 (ActiveMQ-client-global-threads-1566836606)) JTA Crash Record Quickstart: key value pair updated via JMS.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Issue a second XA transaction using your browser at http://eap-app-eap-tx-demo.openshift.example.com/jboss-jta-crash-rec.
-
Enter
Landinto the Key field, andRoverinto the Value field. Click the Submit button. - Wait for a moment, then click the Refresh Table link.
-
Notice how the
Land Roverentry was added without theupdated via …suffix.
-
Enter
Scale the cluster down.
oc scale --replicas=0 dc/eap-app deploymentconfig "eap-app" scaled
$ oc scale --replicas=0 dc/eap-app deploymentconfig "eap-app" scaledCopy to Clipboard Copied! Toggle word wrap Toggle overflow Notice how the
eap-app-2-r00gmpod was scheduled for termination.Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Watch the log of the migration pod and notice how transaction recovery is performed. Wait for the recovery to finish:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Scale the cluster back up.
oc scale --replicas=1 dc/eap-app deploymentconfig "eap-app" scaled
$ oc scale --replicas=1 dc/eap-app deploymentconfig "eap-app" scaledCopy to Clipboard Copied! Toggle word wrap Toggle overflow - Using the browser navigate back to http://eap-app-eap-tx-demo.openshift.example.com/jboss-jta-crash-rec.
Notice the table contains entries for both transactions. It looks similar to the following output:
Expand Table 7.1. Example: Database Table Contents Database Table Contents Key
Value
Mercedes
Benz updated via JMS.
Land
Rover updated via JMS.
The content in the above table indicates that, although the cluster was scaled down before the second XA transaction had chance to finish, the migration pod performed the transaction recovery and the transaction was successfully completed.