Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 2. Planning a deployment of AMQ Broker on OpenShift Container Platform

This section describes how to plan an Operator-based deployment.

Operators are programs that enable you to package, deploy, and manage OpenShift applications. Often, Operators automate common or complex tasks. Commonly, Operators are intended to provide:

  • Consistent, repeatable installations
  • Health checks of system components
  • Over-the-air (OTA) updates
  • Managed upgrades

Operators enable you to make changes while your broker instances are running, because they are always listening for changes to the Custom Resource (CR) instances that you used to configure your deployment. When you make changes to a CR, the Operator reconciles the changes with the existing broker deployment and updates the deployment to reflect the changes. In addition, the Operator provides a message migration capability, which ensures the integrity of messaging data. When a broker in a clustered deployment shuts down due to an intentional scaledown of the deployment, this capability migrates messages to a broker Pod that is still running in the same broker cluster.

2.1. Overview of high availability (HA)
Copy link

The term high availability refers to a system that can remain operational even when part of that system fails or is shut down. For AMQ Broker on OpenShift Container Platform, this means ensuring the integrity and availability of messaging data if a broker pod, node on which a pod is running, or cluster fails.

AMQ Broker uses the HA capabilities provided in OpenShift Container Platform to mitigate pod and node failures:

  • If persistent storage is enabled on AMQ Broker, each broker pod writes its data to a Persistent Volume (PV) that was claimed by using a Persistent Volume Claim (PVC). A PV remains available even after a pod is deleted. If a broker pod fails, OpenShift restarts the pod with the same name and uses the existing PV that contains the messaging data.

  • You can run multiple broker pods in a cluster and distribute pods on separate nodes to recover from a node failure. Each broker pod writes its message data to its own PV which is then available to that broker pod if it is restarted on a different node.

    If the mean time to repair (MTTR) to recover from a node failure on your Openshift cluster does not meet the service availability requirements for AMQ Broker, you can create leader-follower deployments to provide faster recovery. You can also use leader-follower deployments to protect against a cluster or wider data center outage. For more information, see Section 4.23, “Configuring leader-follower broker deployments for high availability”.

Additional resources

For information on how to use persistent storage, see Section 2.9, “Operator deployment notes”.

For information on how to distribute broker pods on separate nodes, see Section 4.17.2, “Controlling pod placement using tolerations”.

In general, a Custom Resource Definition (CRD) is a schema of configuration items that you can modify for a custom OpenShift object deployed with an Operator. By creating a corresponding Custom Resource (CR) instance, you can specify values for configuration items in the CRD. If you are an Operator developer, what you expose through a CRD essentially becomes the API for how a deployed object is configured and used. You can directly access the CRD through regular HTTP curl commands, because the CRD gets exposed automatically through Kubernetes.

You can install the AMQ Broker Operator using either the OpenShift command-line interface (CLI), or the Operator Lifecycle Manager, through the OperatorHub graphical interface. In either case, the AMQ Broker Operator includes the CRDs described below.

Main broker CRD

You deploy a CR instance based on this CRD to create and configure a broker deployment.

Based on how you install the Operator, this CRD is:

  • The broker_activemqartemis_crd file in the crds directory of the Operator installation archive (OpenShift CLI installation method)
  • The ActiveMQArtemis CRD in the Custom Resource Definitions section of the OpenShift Container Platform web console (OperatorHub installation method)
Address CRD

You deploy a CR instance based on this CRD to create addresses and queues for a broker deployment.

Based on how you install the Operator, this CRD is:

  • The broker_activemqartemisaddress_crd file in the crds directory of the Operator installation archive (OpenShift CLI installation method)
  • The ActiveMQArtemisAddresss CRD in the Custom Resource Definitions section of the OpenShift Container Platform web console (OperatorHub installation method)
Note

The address CRD is deprecated in 7.12. You can use the brokerProperties attribute in an ActiveMQArtemis CR instance instead of creating a CR instance based on the addresss CRD.

Security CRD

You deploy a CR instance based on this CRD to create users and associate those users with security contexts.

Based on how you install the Operator, this CRD is:

  • The broker_activemqartemissecurity_crd file in the crds directory of the Operator installation archive (OpenShift CLI installation method)
  • The ActiveMQArtemisSecurity CRD in the Custom Resource Definitions section of the OpenShift Container Platform web console (OperatorHub installation method).
Note

The security CRD is deprecated in 7.12. You can use the brokerProperties attribute in an ActiveMQArtemis CR instance instead of creating a CR instance based on the security CRD.

Scaledown CRD

The Operator automatically creates a CR instance based on this CRD when instantiating a scaledown controller for message migration.

Based on how you install the Operator, this CRD is:

  • The broker_activemqartemisscaledown_crd file in the crds directory of the Operator installation archive (OpenShift CLI installation method)
  • The ActiveMQArtemisScaledown CRD in the Custom Resource Definitions section of the OpenShift Container Platform web console (OperatorHub installation method).
Note

The scaledown CRD is deprecated in 7.12 and is not required to scale down a cluster.

Additional resources

The AMQ Broker Operator archive that you download and extract during installation includes sample Custom Resource (CR) files in the deploy/crs directory. These sample CR files enable you to:

  • Deploy a minimal broker without SSL or clustering.
  • Define addresses.

The broker Operator archive that you download and extract also includes CRs for example deployments in the deploy/examples/address and deploy/examples/artemis directories, as listed below.

address_queue.yaml
Deploys an address and queue with different names. Deletes the queue when the CR is undeployed.
address_topic.yaml
Deploys an address with a multicast routing type. Deletes the address when the CR is undeployed.
artemis_address_settings.yaml
Deploys a broker with specific address settings.
artemis_cluster_persistence.yaml
Deploys clustered brokers with persistent storage.
artemis_enable_metrics_plugin.yaml
Enables the Prometheus metrics plugin to collect metrics.
artemis_resources.yaml
Sets CPU and memory resource limits for the broker.
artemis_single.yaml
Deploys a single broker.

You can use the brokerProperties attribute in an ActiveMQArtemis custom resource to configure any configuration setting for a broker. Using brokerProperties is particularly useful if you want to configure settings that:

  • are not exposed in the ActiveMQArtemis CRD
  • are exposed in the ActiveMQArtemisAddress and ActiveMQArtemisSecurity CRDs.
Note

Both the ActiveMQArtemisAddress and ActiveMQArtemisSecurity CRDs are deprecated starting in AMQ Broker 7.12

Configuration settings added under a brokerProperties attribute are stored in a secret. This secret is mounted as a properties file on the broker pod. At startup, the properties file is applied directly to the internal java configuration bean after the XML configuration is applied.

Examples
In the following example, a single property is applied to the configuration bean. 
spec:
  ...
  brokerProperties:
  - globalMaxSize=500m
  ...
Copy to Clipboard Toggle word wrap

In the following example, multiple properties are applied to nested collections of configuration beans to create a broker connection named target that mirror messages with another broker. 

spec:
  ...
  brokerProperties
  - "AMQPConnections.target.uri=tcp://<hostname>:<port>"
  - "AMQPConnections.target.connectionElements.mirror.type=MIRROR"
  - "AMQPConnections.target.connectionElements.mirror.messageAcknowledgements=true"
  - "AMQPConnections.target.connectionElements.mirror.queueCreation=true"
  - "AMQPConnections.target.connectionElements.mirror.queueRemoval=true"
  ...
Copy to Clipboard Toggle word wrap
Important

Using the brokerProperties attribute provides access to many configuration items that you cannot otherwise configure for AMQ Broker on OpenShift Container Platform. If used incorrectly, some properties can have serious consequences for your deployment. Always exercise caution when configuring the broker using this method.

Status reporting for brokerProperties

The status of items configured in a brokerProperties attribute is provided in the BrokerPropertiesApplied status section of the ActiveMQArtemis CR. For example: 

- lastTransitionTime: "2023-02-06T20:50:01Z"
  message: ""
  reason: Applied
  status: "True"
  type: BrokerPropertiesApplied
Copy to Clipboard Toggle word wrap

The reason field contains one of the following values to show the status of the items configured in a brokerProperties attribute:

Applied
OpenShift Container Platform propagated the updated secret to the properties file on each broker pod.
AppliedWithError
OpenShift Container Platform propagated the updated secret to the properties file on each broker pod. However, an error was found in the brokerProperties configuration. In the status section of the CR, check the message field to identify the invalid property and correct it in the CR.
OutOfSync
OpenShift Container Platform has not yet propagated the updated secret to the properties file on each broker pod. When OpenShift Container Platform propagates the updated secret to each pod, the reason field value changes to Applied.
Note

The broker checks periodically for configuration changes, including updates to the properties file that is mounted on the pod, and reloads the configuration if it detects any changes. However, updates to properties that are read only when the broker starts, for example, JVM settings, are not reloaded until you restart the broker. For more information about which properties are reloaded, see Reloading configuration updates in Configuring AMQ Broker.

Additional Information

For a list of properties that you can configure in the brokerProperties element in a CR, see Broker Properties in Configuring AMQ Broker.

When the Cluster Operator is running, it starts to watch for updates of AMQ Broker custom resources (CRs).

You can choose to deploy the Cluster Operator to watch CRs from:

  • A single namespace (the same namespace containing the Operator)
  • All namespaces
Note

If you have already installed a previous version of the AMQ Broker Operator in a namespace on your cluster, Red Hat recommends that you do not install the AMQ Broker Operator 7.12 version to watch that namespace to avoid potential conflicts.

In the ActiveMQArtemis CR, you can use any of the following configurations to deploy container images:

  • Specify a version number in the spec.version attribute and allow the Operator to choose the broker and init container images to deploy for that version number.
  • Specify the registry URLs of the specific broker and init container images that you want the Operator to deploy in the spec.deploymentPlan.image and spec.deploymentPlan.initImage attributes.
  • Set the value of the spec.deploymentPlan.image attribute to placeholder, which means that the Operator chooses the latest broker and init container images that are known to the Operator version.
Note

If you do not use any of these configurations to deploy container images, the Operator chooses the latest broker and init container images that are known to the Operator version.

After you save a CR, the Operator performs the following validation to determine the configuration to use.

  • The Operator checks if the CR contains a spec.version attribute.

    • If the CR does not contain a spec.version attribute, the Operator checks if the CR contains a spec.deploymentPlan.image and a spec.deployment.Plan.initImage attribute.

      • If the CR contains a spec.deploymentPlan.image and a spec.deployment.Plan.initImage attribute, the Operator deploys the container images that are identified by their registry URLs.
      • If the CR does not contain a spec.deploymentPlan.image and a spec.deployment.Plan.initImage attribute, the Operator chooses the container images to deploy. For more information, see Section 2.7, “How the Operator chooses container images”.

    • If the CR contains a spec.version attribute, the Operator verifies that the version number specified is within the valid range of versions that the Operator supports.

      • If the value of the spec.version attribute is not valid, the Operator stops the deployment.

      • If the value of the spec.version attribute is valid, the Operator checks if the CR contains a spec.deploymentPlan.image and a spec.deployment.Plan.initImage attribute.

        • If the CR contains a spec.deploymentPlan.image and a spec.deployment.Plan.initImage attribute, the Operator deploys the container images that are identified by their registry URLs.
        • If the CR does not contain a spec.deploymentPlan.image and a spec.deployment.Plan.initImage attribute, the Operator chooses the container images to deploy. For more information, see Section 2.7, “How the Operator chooses container images”.
Note

If the CR contains only one of the spec.deploymentPlan.image and the spec.deployment.Plan.initImage attributes, the Operator uses the spec.version number attribute to choose an image for the attribute that is not in the CR, or chooses the latest known image for that attribute if the spec.version attribute is not in the CR.

Red Hat recommends that you do not specify the spec.deploymentPlan.image attribute without the spec.deployment.Plan.initImage attribute, or vice versa, to prevent mismatched versions of broker and init container images from being deployed.

2.7. How the Operator chooses container images
Copy link

If a CR does not contain a spec.deploymentPlan.image and a spec.deployment.Plan.initImage attribute, which specify the registry URLs of specific container images the Operator must deploy, the Operator automatically chooses the appropriate container images to deploy.

Note

If you install the Operator using the OpenShift command-line interface, the Operator installation archive includes a sample CR file called broker_activemqartemis_cr.yaml. In the sample CR, the spec.deploymentPlan.image property is included and set to its default value of placeholder. This value indicates that the Operator does not choose a broker container image until you deploy the CR.

The spec.deploymentPlan.initImage property, which specifies the Init Container image, is not included in the broker_activemqartemis_cr.yaml sample CR file. If you do not explicitly include the spec.deploymentPlan.initImage property in your CR and specify a value, the Operator chooses a built-in Init Container image that matches the version of the Operator container image chosen.

To choose broker and Init Container images, the Operator first determines an AMQ Broker version of the images that is required. The Operator gets the version from the value of the spec.version property. If the spec.version property is not set, the Operator uses the latest version of the images for AMQ Broker.

The Operator then detects your container platform. The AMQ Broker Operator can run on the following container platforms:

  • OpenShift Container Platform (x86_64)
  • OpenShift Container Platform on IBM Z (s390x)
  • OpenShift Container Platform on IBM Power Systems (ppc64le)

Based on the version of AMQ Broker and your container platform, the Operator then references two sets of environment variables in the operator.yaml configuration file. These sets of environment variables specify broker and Init Container images for various versions of AMQ Broker, as described in the following section.

The environment variables included in the operator.yaml have the following naming convention.

Expand
Table 2.1. Naming conventions for environment variables
Container platformNaming convention

OpenShift Container Platform

RELATED_IMAGE_ActiveMQ_Artemis_Broker_Kubernetes_<AMQ_Broker_version>

OpenShift Container Platform on IBM Z

RELATED_IMAGE_ActiveMQ_Artemis_Broker_Kubernetes_<AMQ_Broker_version>_s390x

OpenShift Container Platform on IBM Power Systems

RELATED_IMAGE_ActiveMQ_Artemis_Broker_Kubernetes_<AMQ_Broker_version>_ppc64le

The following are examples of environment variable names for broker and init container images for each supported container platform.

Expand
Table 2.2. Example environment variable names
Container platformEnvironment variable names

OpenShift Container Platform

RELATED_IMAGE_ActiveMQ_Artemis_Broker_Kubernetes_7126
RELATED_IMAGE_ActiveMQ_Artemis_Broker_Init_7126

OpenShift Container Platform on IBM Z

RELATED_IMAGE_ActiveMQ_Artemis_Broker_Kubernetes_7126_s390x
RELATED_IMAGE_ActiveMQ_Artemis_Broker_Init_s390x_7126

OpenShift Container Platform on IBM Power Systems

RELATED_IMAGE_ActiveMQ_Artemis_Broker_Kubernetes_7126_ppc64le
RELATED_IMAGE_ActiveMQ_Artemis_Broker_Init_ppc64le_7126

The value of each environment variable specifies the address of a container image that is available from Red Hat. The image name is represented by a Secure Hash Algorithm (SHA) value. For example: 

- name: RELATED_IMAGE_ActiveMQ_Artemis_Broker_Kubernetes_7126
  value: registry.redhat.io/amq7/amq-broker-rhel8@sha256:7079f8cde9cdbf42cf7d0b446dfd45a65ecd139f527e9f8c27ab35037cf118f8
Copy to Clipboard Toggle word wrap

Therefore, based on an AMQ Broker version and your container platform, the Operator determines the applicable environment variable names for the broker and init container. The Operator uses the corresponding image values when starting the broker container.

Additional resources

After you save a CR, the Operator performs the following validation of the CR configuration and provides a status in the CR.

Expand
Table 2.3. Operator validation of CR configuration
ValidationPurpose of validationStatus reported in CR

Does the CR contain a spec.deploymentPlan.image attribute without a spec.version attribute.

A spec.deploymentPlan.image attribute without a spec.version attribute causes the Operator to restart the broker pods each time the Operator is upgraded. Pod restarts are required because the new Operator updates a label in the StatefulSet with the latest supported broker version unless a version number is explicitly set in the spec.version attribute.

The Valid condition is Unknown and the following status message is displayed: Unknown image version, set a supported broker version in spec.version when images are specified.

Does the CR contain a spec.deploymentPlan.image attribute without a spec.deploymentPlan.initImage attribute or vice versa.

With this configuration, different versions of the broker and init container images could be deployed, which might prevent your broker from starting.

The`Valid` condition is Unknown and the following status message is displayed: Init image and broker image must both be configured as an interdependent pair.

If the CR contain a spec.version attribute, is the version specified within the range of versions that the Operator supports.

If the value of the spec.version attribute is a broker version that is not supported by the Operator, the Operator does not proceed with the deployment of broker pods.

The Valid condition is False and the following status message is displayed: Spec.Version does not resolve to a supported broker version, reason did not find a matching broker in the supported list for <version>.

Does the version of the broker image deployed, based on the URL of a container image in the spec.deploymentPlan.image attribute, match the broker version in the spec.version attribute.

Flags a mismatch between the actual broker version deployed and the version shown in the spec.version attribute if both attributes are configured in the CR. This is for information purposes to highlight that the version shown in the spec.version attribute is not the version deployed.

The status of the BrokerVersionAligned condition is Unknown and the following message is displayed: broker version non aligned on pod <pod name>, the detected version <version> doesn’t match the spec.version <version> resolved as <version>.

Additional resources

For more information on viewing status information in a CR, see Viewing status information for your broker deployment.

2.9. Operator deployment notes
Copy link

This section describes some important considerations when planning an Operator-based deployment

  • Deploying the Custom Resource Definitions (CRDs) that accompany the AMQ Broker Operator requires cluster administrator privileges for your OpenShift cluster. When the Operator is deployed, non-administrator users can create broker instances via corresponding Custom Resources (CRs). To enable regular users to deploy CRs, the cluster administrator must first assign roles and permissions to the CRDs. For more information, see Creating cluster roles for Custom Resource Definitions in the OpenShift Container Platform documentation.
  • When you update your cluster with the CRDs for the latest Operator version, this update affects all projects in the cluster. Any broker pods deployed from previous versions of the Operator might become unable to update their status. When you click the Logs tab of a running broker pod in the OpenShift Container Platform web console, you see messages indicating that 'UpdatePodStatus' has failed. However, the broker pods and Operator in that project continue to work as expected. To fix this issue for an affected project, you must also upgrade that project to use the latest version of the Operator.

  • While you can create more than one broker deployment in a given OpenShift project by deploying multiple Custom Resource (CR) instances, typically, you create a single broker deployment in a project, and then deploy multiple CR instances for addresses.

    Red Hat recommends you create broker deployments in separate projects.

  • If you intend to deploy brokers with persistent storage and do not have container-native storage in your OpenShift cluster, you need to manually provision Persistent Volumes (PVs) and ensure that these are available to be claimed by the Operator. For example, if you want to create a cluster of two brokers with persistent storage (that is, by setting persistenceEnabled=true in your CR), you need to have two persistent volumes available. By default, each broker instance requires storage of 2 GiB.

    If you specify persistenceEnabled=false in your CR, the deployed brokers uses ephemeral storage. Ephemeral storage means that that every time you restart the broker pods, any existing data is lost.

    For more information about provisioning persistent storage in OpenShift Container Platform, see:

  • You must add configuration for the items listed below to the main broker CR instance before deploying the CR for the first time. You cannot add configuration for these items to a broker deployment that is already running.

  • If you update a parameter in your CR that the Operator is unable to dynamically update in the StatefulSet, the Operator deletes the StatefulSet and recreates it with the updated parameter value. Deleting the StatefulSet causes all pods to be deleted and recreated, which causes a temporary broker outage. An example of a CR update that the Operator cannot dynamically update in the StatefulSet is if you change persistenceEnabled=false to persistenceEnabled=true.

If the cluster already contains installed Operators for AMQ Broker, and you want a new Operator to watch all or multiple namespaces, you must ensure that the new Operator does not watch any of the same namespaces as existing Operators. Use the following procedure to identify the namespaces watched by existing Operators.

Procedure

  1. In the left pane of the OpenShift Container Platform web console, click Workloads Deployments.
  2. In the Project drop-down list, select All Projects.

  3. In the Filter Name box, specify a string, for example, amq, to display the Operators for AMQ Broker that are installed on the cluster.

    Note

    The namespace column displays the namespace where each operator is deployed.

  4. Check the namespaces that each installed Operator for AMQ Broker is configured to watch.

    1. Click the Operator name to display the Operator details and click the YAML tab.

    2. Search for WATCH_NAMESPACE and note the namespaces that the Operator watches.

      • If the WATCH_NAMESPACE section has a fieldPath field that has a value of metadata.namespace, the Operator is watching the namespace where it is deployed.

      • If the WATCH_NAMESPACE section has a value field that has list of namespaces, the Operator is watching the specified namespaces. For example: 

        - name: WATCH_NAMESPACE
  value: "namespace1, namespace2"
        Copy to Clipboard Toggle word wrap

      • If the WATCH_NAMESPACE section has a value field that is empty or has an asterisk, the Operator is watching all the namespaces on the cluster. For example: 

        - name: WATCH_NAMESPACE
  value: ""
        Copy to Clipboard Toggle word wrap

        In this case, before you deploy the new Operator, you must either uninstall the existing Operator or reconfigure it to watch specific namespaces.

The procedures in the next section show you how to install the Operator and use Custom Resources (CRs) to create broker deployments on OpenShift Container Platform. After you complete the procedures, the Operator runs in an individual Pod and each broker instance that you create runs as an individual Pod in a StatefulSet in the same project as the Operator. Later, you will see how to use a dedicated addressing CR to define addresses in your broker deployment.

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top