Chapter 6. Upgrading an Operator-based broker deployment

Procedure

1. Log in to the OpenShift Container Platform web console as a cluster administrator.
2. Uninstall the existing AMQ Broker Operator from your project.
3. In the left navigation menu, click Operators Installed Operators.
4. From the Project drop-down menu at the top of the page, select the project in which you want to uninstall the Operator.
5. Locate the Red Hat Integration - AMQ Broker instance that you want to uninstall.

6. For your Operator instance, click the More Options icon (three vertical dots) on the right-hand side. Select Uninstall Operator.

   Warning

   Ensure that the Delete all operand instances for this operator checkbox is NOT selected. If this checkbox is selected, the broker instances that are managed by the Operator are deleted when the Operator is uninstalled.

7. On the confirmation dialog box, click Uninstall.
8. Use OperatorHub to install the latest version of the Operator for AMQ Broker 7.12. For more information, see Section 3.3.2, “Deploying the Operator from OperatorHub”.
9. In the left navigation menu, click Operators Installed Operators.
10. Verify that the correct version number is displayed under the Red Hat Integration - AMQ Broker instance that is installed.

Procedure

1. Log in to the OpenShift Container Platform web console as a cluster administrator.

2. Uninstall the existing AMQ Broker Operator from your project.

   1. In the left navigation menu, click Operators Installed Operators.
   2. From the Project drop-down menu at the top of the page, select the project in which you want to uninstall the Operator.
   3. Locate the Red Hat Integration - AMQ Broker instance that you want to uninstall.
   4. For your Operator instance, click the More Options icon (three vertical dots) on the right-hand side. Select Uninstall Operator.
   5. On the confirmation dialog box, click Uninstall.

3. When you upgrade a 7.10.0 Operator, the new Operator deletes the StatefulSet to remove custom and Operator metering labels, which were incorrectly added to the StatefulSet selector by the Operator in 7.10.0. When the Operator deletes the StatefulSet, it also deletes the existing broker pods, which causes a temporary broker outage. If you want to avoid the outage, complete the following steps to delete the StatefulSet and orphan the broker pods so that they continue to run.

   1. Log in to OpenShift Container Platform CLI as an administrator for the project that contains your existing Operator deployment:

      $ oc login -u <user>

      Copy to Clipboard Toggle word wrap

   2. Switch to the OpenShift project in which you want to upgrade your Operator version.

      $ oc project <project-name>

      Copy to Clipboard Toggle word wrap

   3. Delete the StatefulSet with the --cascade=orphan option to orphan the broker pods. The orphaned broker pods continue to run after the StatefulSet is deleted.

      $ oc delete statefulset <statefulset-name> --cascade=orphan

      Copy to Clipboard Toggle word wrap

4. Check if your main broker CR has labels called application or ActiveMQArtemis configured in the deploymentPlan.labels attribute.

   In 7.10.0, it was possible to configure these custom labels in the CR. These labels are reserved for the Operator to assign labels to pods and cannot be added as custom labels after 7.10.0. If these custom labels were configured in the main broker CR in 7.10.0, the Operator-assigned labels on the pods were overwritten by the custom labels. If the CR has either of these labels, complete the following steps to restore the correct labels on the pods and remove the labels from the CR.

   1. In the OpenShift command-line interface (CLI), run the following command to restore the correct pod labels. In the following example, 'ex-aao' is the name of the StatefulSet deployed.

      $ for pod in $(oc get pods | grep -o '^ex-aao[^ ]*'); do oc label --overwrite pods $pod ActiveMQArtemis=ex-aao application=ex-aao-app; done

      Copy to Clipboard Toggle word wrap

   2. Delete the application and ActiveMQArtemis labels from the deploymentPlan.labels attribute in the CR.

      1. Using the OpenShift command-line interface:

         1. Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment.

            oc login -u <user> -p <password> --server=<host:port>

            Copy to Clipboard Toggle word wrap

         2. Edit the CR for your deployment.

            oc edit ActiveMQArtemis <statefulset name> -n <namespace>

            Copy to Clipboard Toggle word wrap
         3. In the deploymentPlan.labels element in the CR, delete any custom labels called application or ActiveMQArtemis.
         4. Save the CR.

      2. Using the OpenShift Container Platform web console:

         1. Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment.
         2. In the left pane, click Administration Custom Resource Definitions.
         3. Click the ActiveMQArtemis CRD.
         4. Click the Instances tab.
         5. Click the instance for your broker deployment.

         6. Click the YAML tab.

            Within the console, a YAML editor opens, enabling you to configure a CR instance.

         7. In the deploymentPlan.labels element in the CR, delete any custom labels called application or ActiveMQArtemis.
         8. Click Save.

5. Use OperatorHub to install the latest version of the Operator for AMQ Broker 7.12. For more information, see Section 3.3.2, “Deploying the Operator from OperatorHub”.

   The new Operator can recognize and manage your previous broker deployments. If you set values in the image or version field in the CR, the Operator’s reconciliation process upgrades the broker pods to the corresponding images when the Operator starts. For more information, see Section 6.6, “Restricting automatic upgrades of broker container images”. Otherwise, the Operator upgrades each broker pod to the latest container image.

   Note

   If the reconciliation process does not start, you can start the process by scaling the deployment. For more information, see Section 3.4.1, “Deploying a basic broker instance”.

6. Add attributes to the CR for the new features that are available in the upgraded broker, as required.

Procedure

1. Log in to the OpenShift Container Platform web console as a cluster administrator.

2. Check if your main broker CR has labels called application or ActiveMQArtemis configured in the deploymentPlan.labels attribute.

   These labels are reserved for the Operator to assign labels to pods and cannot be used after 7.10.1. If these custom labels were configured in the main broker CR, the Operator-assigned labels on the pods were overwritten by the custom labels.

3. If these custom labels are not configured in the main broker CR, use OperatorHub to install the latest version of the Operator for AMQ Broker 7.12. For more information, see Section 3.3.2, “Deploying the Operator from OperatorHub”.

4. If either of these custom labels are configured in the main broker CR, complete the following steps to uninstall the existing Operator, restore the correct pod labels and remove the labels from the CR, before you install the new Operator.

   Note

   By uninstalling the Operator, you can remove the custom labels without the Operator deleting the StatefulSet, which also deletes the existing broker pods and causes a temporary broker outage.

   1. Uninstall the existing AMQ Broker Operator from your project.

      1. In the left navigation menu, click Operators Installed Operators.
      2. From the Project drop-down menu at the top of the page, select the project from which you want to uninstall the Operator.
      3. Locate the Red Hat Integration - AMQ Broker instance that you want to uninstall.
      4. For your Operator instance, click the More Options icon (three vertical dots) on the right-hand side. Select Uninstall Operator.
      5. On the confirmation dialog box, click Uninstall.

   2. In the OpenShift command-line interface (CLI), run the following command to restore the correct pod labels. In the following example, 'ex-aao' is the name of the StatefulSet deployed.

      $ for pod in $(oc get pods | grep -o '^ex-aao[^ ]*'); do oc label --overwrite pods $pod ActiveMQArtemis=ex-aao application=ex-aao-app; done

      Copy to Clipboard Toggle word wrap

   3. Delete the application and ActiveMQArtemis labels from the deploymentPlan.labels attribute in the CR.

      1. Using the OpenShift command-line interface:

         1. Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment.

            oc login -u <user> -p <password> --server=<host:port>

            Copy to Clipboard Toggle word wrap

         2. Edit the CR for your deployment.

            oc edit ActiveMQArtemis <statefulset name> -n <namespace>

            Copy to Clipboard Toggle word wrap
         3. In the deploymentPlan.labels attribute in the CR, delete any custom labels called application or ActiveMQArtemis.
         4. Save the CR file.

      2. Using the OpenShift Container Platform web console:

         1. Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment.
         2. In the left pane, click Administration Custom Resource Definitions.
         3. Click the ActiveMQArtemis CRD.
         4. Click the Instances tab.
         5. Click the instance for your broker deployment.

         6. Click the YAML tab.

            Within the console, a YAML editor opens, enabling you to configure a CR instance.

         7. In the deploymentPlan.labels attribute in the CR, delete any custom labels called application or ActiveMQArtemis.
         8. Click Save.

5. Use OperatorHub to install the latest version of the Operator for AMQ Broker 7.12. For more information, see Section 3.3.2, “Deploying the Operator from OperatorHub”.

   The new Operator can recognize and manage your previous broker deployments. If you set values in the image or version field in the CR, the Operator’s reconciliation process upgrades the broker pods to the corresponding images when the Operator starts. For more information, see Section 6.6, “Restricting automatic upgrades of broker container images”. Otherwise, the Operator upgrades each broker pod to the latest container image.

   Note

   If the reconciliation process does not start, you can start the process by scaling the deployment. For more information, see Section 3.4.1, “Deploying a basic broker instance”.

6. Add attributes to the CR for the new features that are available in the upgraded broker, as required.

Procedure

1. In your web browser, navigate to the Software Downloads page for AMQ Broker 7.12.6.
2. Ensure that the value of the Version drop-down list is set to 7.12.6 and the Releases tab is selected.

3. Next to AMQ Broker 7.12.6 Operator Installation and Example Files, click Download.

   Download of the amq-broker-operator-7.12.6-ocp-install-examples-rhel8.zip compressed archive automatically begins.

4. When the download has completed, move the archive to your chosen installation directory. The following example moves the archive to a directory called ~/broker/operator.

   $ mkdir ~/broker/operator
   $ mv amq-broker-operator-7.12.6-ocp-install-examples-rhel8.zip ~/broker/operator

   Copy to Clipboard Toggle word wrap

5. In your chosen installation directory, extract the contents of the archive. For example:

   $ cd ~/broker/operator
   $ unzip amq-broker-operator-operator-7.12.6-ocp-install-examples-rhel8.zip

   Copy to Clipboard Toggle word wrap

6. Log in to OpenShift Container Platform as an administrator for the project that contains your existing Operator deployment.

   $ oc login -u <user>

   Copy to Clipboard Toggle word wrap

7. Switch to the OpenShift project in which you want to upgrade your Operator version.

   $ oc project <project-name>

   Copy to Clipboard Toggle word wrap

8. In the deploy directory of the latest Operator archive that you downloaded and extracted, open the operator.yaml file.

   Note

   In the operator.yaml file, the Operator uses an image that is represented by a Secure Hash Algorithm (SHA) value. The comment line, which begins with a number sign (#) symbol, denotes that the SHA value corresponds to a specific container image tag.

9. Open the operator.yaml file for your previous Operator deployment. Check that any non-default values that you specified in your previous configuration are replicated in the new operator.yaml configuration file.

10. In the new operator.yaml file, the Operator is named amq-broker-controller-manager by default. If the name of the Operator in your previous deployment is not amq-broker-controller-manager, replace all instances of amq-broker-controller-manager with the previous Operator name. For example:

    spec:
      ...
      selector
        matchLabels
          name: amq-broker-operator
      ...

    Copy to Clipboard Toggle word wrap

11. In the new operator.yaml file, the service account for the Operator is named amq-broker-controller-manager. In previous versions, the service account for the Operator was named amq-broker-operator.

    1. If you want to use the service account name in your previous deployment, replace the name of the service account in the new operator.yaml file with the name used in the previous deployment. For example:

       spec:
         ...
         serviceAccountName: amq-broker-operator
         ...

       Copy to Clipboard Toggle word wrap

    2. If you want to use the new service account name, amq-broker-controller-manager for the Operator, update the service account, role, and role binding in your project.

       $ oc apply -f deploy/service_account.yaml

       Copy to Clipboard Toggle word wrap

       $ oc apply -f deploy/role.yaml

       Copy to Clipboard Toggle word wrap

       $ oc apply -f deploy/role_binding.yaml

       Copy to Clipboard Toggle word wrap

12. Update the CRDs that are included with the Operator.

    1. Update the main broker CRD.

       $ oc apply -f deploy/crds/broker_activemqartemis_crd.yaml

       Copy to Clipboard Toggle word wrap

    2. Update the address CRD.

       $ oc apply -f deploy/crds/broker_activemqartemisaddress_crd.yaml

       Copy to Clipboard Toggle word wrap

    3. Update the scaledown controller CRD.

       $ oc apply -f deploy/crds/broker_activemqartemisscaledown_crd.yaml

       Copy to Clipboard Toggle word wrap

    4. Update the security CRD.

       $ oc apply -f deploy/crds/broker_activemqartemissecurity_crd.yaml

       Copy to Clipboard Toggle word wrap

13. If you are upgrading from AMQ Broker Operator 7.10.0 only, delete the Operator and the StatefulSet.

    By default, the new Operator deletes the StatefulSet to remove custom and Operator metering labels, which were incorrectly added to the StatefulSet selector by the Operator in 7.10.0. When the Operator deletes the StatefulSet, it also deletes the existing broker pods, which causes a temporary broker outage. If you want to avoid an outage, complete the following steps to delete the Operator and the StatefulSet without deleting the broker pods.

    1. Delete the Operator.

       $ oc delete -f deploy/operator.yaml

       Copy to Clipboard Toggle word wrap
       Note

       Deleting the Operator does not remove the broker instances that are managed by the Operator.

    2. Delete the StatefulSet with the --cascade=orphan option to orphan the broker pods. The orphaned broker pods continue to run after the StatefulSet is deleted.

       $ oc delete statefulset <statefulset-name> --cascade=orphan

       Copy to Clipboard Toggle word wrap

14. If you are upgrading from AMQ Broker Operator 7.10.0 or 7.10.1, check if your main broker CR has labels called application or ActiveMQArtemis configured in the deploymentPlan.labels attribute.

    These labels are reserved for the Operator to assign labels to pods and are not permitted as custom labels after 7.10.1. If these custom labels were configured in the main broker CR, the Operator-assigned labels on the pods were overwritten by the custom labels. If either of these custom labels are configured in the main broker CR, complete the following steps to restore the correct labels on the pods and remove the labels from the CR.

    1. If you are upgrading from 7.10.0, you deleted the Operator in the previous step. If you are upgrading from 7.10.1, delete the Operator.

       $ oc delete -f deploy/operator.yaml

       Copy to Clipboard Toggle word wrap

    2. Run the following command to restore the correct pod labels. In the following example, 'ex-aao' is the name of the StatefulSet deployed.

       $ for pod in $(oc get pods | grep -o '^ex-aao[^ ]*'); do oc label --overwrite pods $pod ActiveMQArtemis=ex-aao application=ex-aao-app; done

       Copy to Clipboard Toggle word wrap

    3. Delete the application and ActiveMQArtemis labels from the deploymentPlan.labels attribute in the CR.

       1. Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment.

          oc login -u <user> -p <password> --server=<host:port>

          Copy to Clipboard Toggle word wrap
       2. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted.
       3. In the deploymentPlan.labels attribute in the CR, delete any custom labels called application or ActiveMQArtemis.
       4. Save the CR file.

       5. Deploy the CR instance.

          1. Switch to the project for the broker deployment.

             $ oc project <project_name>

             Copy to Clipboard Toggle word wrap

          2. Apply the CR.

             $ oc apply -f <path/to/broker_custom_resource_instance>.yaml

             Copy to Clipboard Toggle word wrap

    4. If you deleted the previous Operator, deploy the new Operator.

        $ oc create -f deploy/operator.yaml

       Copy to Clipboard Toggle word wrap

15. Apply the updated Operator configuration.

    $ oc apply -f deploy/operator.yaml

    Copy to Clipboard Toggle word wrap
16. Add attributes to the CR for the new features that are available in the upgraded broker, as required.

Procedure

1. Edit the main broker CR instance for the broker deployment.

   1. Using the OpenShift command-line interface:

      1. Log in to OpenShift as a user that has privileges to edit and deploy CRs in the project for the broker deployment.

         $ oc login -u <user> -p <password> --server=<host:port>

         Copy to Clipboard Toggle word wrap

      2. Edit the CR.

          oc edit ActiveMQArtemis <CR instance name> -n <namespace>

         Copy to Clipboard Toggle word wrap

   2. Using the OpenShift Container Platform web console:

      1. Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment.
      2. In the left pane, click Operators Installed Operator.
      3. Click the Red Hat Integration - AMQ Broker for RHEL 8 (Multiarch) operator.
      4. Click the AMQ Broker tab.
      5. Click the name of the ActiveMQArtemis instance name.

      6. Click the YAML tab.

         Within the console, a YAML editor opens, enabling you to edit the CR instance.

         Note

         In the status section of the CR, the .status.version.brokerVersion field shows the version of AMQ Broker that is currently deployed.

2. In the spec.version attribute, specify the version to which the Operator can upgrade the broker and init container images in your deployment. The following are examples of values that you can specify.

   Examples

   In the following example, the Operator upgrades the current container images in your deployment to 7.12.0.

   spec:
      version: '7.12.0'
       ...

   Copy to Clipboard Toggle word wrap

   In the following example, the Operator upgrades the current container images in your deployment to the latest available 7.11.x images. For example, if your deployment is using 7.11.1 container images, the Operator automatically upgrades the images to 7.11.6 but not to 7.12.6.

   spec:
       version: '7.11'
       ...

   Copy to Clipboard Toggle word wrap

   In the following example, the Operator upgrades the current container images in your deployment to the latest 7.x.x images. For example, if your deployment is using 7.11.6 images, the Operator automatically upgrades the images to 7.12.6.

   spec:
       version: '7'
       ...

   Copy to Clipboard Toggle word wrap

   Note

   To upgrade between minor versions of the container images, for example, from 7.11.x to 7.12.x, you require an Operator that has the same minor version as that of the new container images. For example, to upgrade from 7.11.6 to 7.12.6, a 7.12.x Operator must be installed.

3. Save the CR.

Procedure

1. Obtain the URLs of the broker and init container images to which the Operator can upgrade the current images.

   1. In the Red Hat Catalog, open the broker container component page: AMQ Broker for RHEL 8 (Multiarch).
   2. In the Architecture drop-down, select your architecture.
   3. In the Tag drop-down, select the tag that corresponds to the image you want to install. Tags are displayed in chronological order based on the release date. A tag consists of the release version and an assigned tag.
   4. Open the Get this image tab.
   5. In the Manifest field, click the Copy icon.
   6. Paste the URL into a text file.
   7. In the Red Hat Catalog, open the init container component page: AMQ Broker Init for RHEL 8 (Multiarch)
   8. To obtain the URL of the init container image, repeat the steps that you followed to obtain the URL of the broker container image.

2. Edit the main broker CR instance for the broker deployment.

   1. Using the OpenShift command-line interface:

      1. Log in to OpenShift as a user that has privileges to edit and deploy CRs in the project for the broker deployment.

         $ oc login -u <user> -p <password> --server=<host:port>

         Copy to Clipboard Toggle word wrap

      2. Edit the CR.

          oc edit ActiveMQArtemis <CR instance name> -n <namespace>

         Copy to Clipboard Toggle word wrap

   2. Using the OpenShift Container Platform web console:

      1. Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment.
      2. In the left pane, click Operators Installed Operator.
      3. Click the Red Hat Integration - AMQ Broker for RHEL 8 (Multiarch) operator.
      4. Click the AMQ Broker tab.
      5. Click the name of the ActiveMQArtemis instance name.

      6. Click the YAML tab.

         Within the console, a YAML editor opens, which enables you to configure the CR instance

   3. Copy the URLs of the broker and init container images that you recorded in the text file and insert them in the spec.deploymentPlan.image and spec.deploymentPlan.initImage fields in the CR. For example:

      spec:
        ...
        deploymentPlan:
          image: registry.redhat.io/amq7/amq-broker-rhel8@7079f8cde9cdbf42cf7d0b446dfd45a65ecd139f527e9f8c27ab35037cf118f8
          initImage: registry.redhat.io/amq7/amq-broker-init-rhel8@d0a1943ae856049327266f60c486f0af7ffa26ba9cf3b674f6be369064531202
        ...

      Copy to Clipboard Toggle word wrap

3. Save the CR.

   When you save the CR, the Operator upgrades the brokers to use the new images and uses these images until you update the values of the spec.deploymentPlan.image and spec.deploymentPlan.initImage attributes again.

4. If you want to prevent future Operator upgrades from restarting the brokers in your deployment, edit the CR and and specify the version number of the brokers that are deployed in the spec.version attribute.

   If the spec.version attribute is not configured in the CR, subsequent upgrades of the Operator cause the broker pods to restart. The pod restart is required because the new Operator adds the latest supported broker version to a label in the StatefulSet unless a version number is explicitly set in the spec.version attribute.

   You can find the version number value to specify for the spec.version attribute in the status section of the CR after the brokers start. For more information, see Viewing status information for your broker deployment.