Release notes

To learn more about observability, see Observing environments introduction.

When an OpenShift Container Platform cluster is in the upgrade stage, the cluster pods are restarted and the cluster might remain in upgrade failed status for a variation of 1-5 minutes. This behavior is expected and resolves after a few minutes.

Certificate manager must not exist on a cluster when you install Red Hat Advanced Cluster Management for Kubernetes.

When certificate manager already exists on the cluster, Red Hat Advanced Cluster Management for Kubernetes installation fails.

To resolve this issue, verify if the certificate manager is present in your cluster by running the following command:

kubectl get crd | grep certificates.certmanager

You might see a discrepancy between the nodes dispalyed on the Cluster page and the Search results.

LDAP user names are case-sensitive. You must use the name exactly the way it is configured in your LDAP directory.

The product supports Mozilla Firefox 74.0 or the latest version that is available for Linux, macOS, and Windows. Upgrade to the latest version for the best console compatibility.

From the console and Visual Web Terminal, users are unable to search for values that contain an empty space.

When you are logged in as kubeadmin and you click the Log out option in the drop-down menu, the console returns to the login screen, but a browser tab opens with a /logout URL. The page is blank and you can close the tab without impact to your console.

For security reasons, search does not display the contents of secrets found on managed clusters. When you search for a secret from the console, you might receive the following error message:

Unable to load resource data - Check to make sure the cluster hosting this resource is online

When you update the storage size in the searchcustomization CR, the PVC configuration does not change. If you need to update the storage size, update the PVC (<storageclassname>-search-redisgraph-0) with the following command:

oc edit pvc <storageclassname>-search-redisgraph-0

When you use the Safari v14.0.3 web browser, the YAML file is not displayed from the Search page. See Supported browsers for other supported browsers or upgrade your Safari version.

The redisgraph StatefulSet and pod are not refreshed when you upgrade from 2.2.z to 2.2.5. You must delete the redisgraph StatefulSet manually, so that the changes are picked up. Complete the following steps to fix this issue:

The redisgraph StatefulSet and pod have restarted.

The observability endpoint operator fails if you create a pull-secret to deploy to the MultiClusterObservability CustomResource (CR) and there is no pull-secret in the open-cluster-management-observability namespace. When you import a new cluster, or import a Hive cluster that is created with Red Hat Advanced Cluster Management, you need to manually create a pull-image secret on the managed cluster.

For more information, see Enabling observability.

When the managed cluster is detached forcefully, the ObservabilityAddon resource (observability-addon) in the cluster namespace is stuck in the Terminating status and cannot be removed. Also, the cluster namespace cannot be deleted.

To fix this problem, you can update the ObervabilityAddon resource in the cluster namespace. Update the resource by deleting the finalizers parameter in the metadata. Run the following command:

kubectl edit observabilityaddon observability-addon -n <CLUSTER_NAMESPACE>

CLUSTER_NAMESPACE is the cluster namespace for the detached cluster.

After the finalizers parameter is removed, the ObervabilityAddon resource is removed.

Red Hat Advanced Cluster Management observability does not display data from an ROKS cluster on some panels within built-in dashboards. This is because ROKS does not expose any API Server metrics from servers they manage. The following Grafana dashboards contain panels that do not support ROKS clusters: Kubernetes/API server, Kubernetes/Compute Resources/Workload, Kubernetes/Compute Resources/Namespace(Workload)

After you upgrade Red Hat Advanced Cluster Management from 2.2.3 to 2.2.4, you might notice that the metrics-collector in the open-cluster-management-addon-observability namespace stops collecting data from your managed clusters. This is because the image manifests ConfigMap was upgraded after the multicluster observability operator was upgraded.

You might see the following Quay.io images being used in the endpoint-observabililty-operator YAML file: quay.io/open-cluster-management/endpoint-monitoring-operator:2.2.0-6a5ea47fc39d51fb4fade6157843f2977442996e and quay.io/open-cluster-management/metrics-collector:2.2.0-ff79e6ec8783756b942a77f08b3ab763dfd2dc15.

To fix this issue, delete multicluster-observability-operator pods that are in the open-cluster-management namespace in the hub cluster. After the new pod is created, a new endpoint-observability-operator deployment is created in your managed cluster with the correct images.

After you upgrade Red Hat Advanced Cluster Management from 2.1 to 2.2.x, the MultiClusterObservability custom resource (CR) continues to display Installing from the OpenShift Container Platform console. You can confirm that MultiClusterObservability is deployed successfully by ensuring that the metrics data is displayed from the Grafana console.

Some metric data is not displayed on the Grafana dashboard for Red Hat OpenShift Container Platform verson 4.8 clusters. You must upgrade to version 2.3. For more information, see Upgrading by using the operator.

When you try to provision a cluster on Google Cloud Platform (GCP), it might fail with the following error:

Cluster initialization failed because one or more operators are not functioning properly.
The cluster should be accessible for troubleshooting as detailed in the documentation linked below,
https://docs.openshift.com/container-platform/latest/support/troubleshooting/troubleshooting-installations.html
The 'wait-for install-complete' subcommand can then be used to continue the installation

You can work around this error by enabling the Network Security API on the GCP project, which allows your cluster installation to continue.

When you upgrade a managed cluster or a local cluster on the Cluster details page, it might take up to 10 minutes after the upgrade process completes for the updated version number to display on the Cluster details page.

You cannot create bare metal managed clusters by using the Red Hat Advanced Cluster Management hub cluster when the hub cluster is hosted on OpenShift Container Platform version 4.7.

When you detach a managed cluster, the Create resources page might temporarily break and display the following error:

Error occurred while retrieving clusters info. Not found.

Wait until the namespace automatically gets removed, which takes 5-10 minutes after you detach the cluster. Or, if the namespace is stuck in a terminating state, you need to manually delete the namespace. Return to the page to see if the error resolved.

Hub cluster and manage cluster time might become out-of-sync, displaying in the console unknown and eventually available within a few minutes. Ensure that the Red Hat OpenShift Container Platform hub cluster time is configured correctly. See Customizing nodes.

After a cluster is imported, log in to the imported cluster and make sure all pods that are deployed by the Klusterlet are running. Otherwise, you might see inconsistent data in the console.

For example, if a policy controller is not running, you might not get the same results of violations on the Governance and risk page and the Cluster status.

For instance, you might see 0 violations listed in the Overview status, but you might have 12 violations reported on the Governance and risk page.

In this case, inconsistency between the pages represents a disconnection between the policy-controller-addon on managed clusters and the policy controller on the hub cluster. Additionally, the managed cluster might not have enough resources to run all the Klusterlet components.

As a result, the policy was not propagated to managed cluster, or the violation was not reported back from managed clusters.

When you import a cluster that was previously managed and detached by a Red Hat Advanced Cluster Management hub cluster, the import process might fail the first time. The cluster status is pending import. Run the command again, and the import should be successful.

You cannot import IBM Red Hat OpenShift Kubernetes Service version 3.11 clusters. Later versions of IBM OpenShift Kubernetes Service are supported.

When you change your cloud provider access key, the provisioned cluster access key is not updated in the namespace. This is required when your credentials expire on the cloud provider where the managed cluster is hosted and you try delete the managed cluster. If something like this occurs, run the following command for your cloud provider to update the access key:

You must be logged in as root to run the management-ingress service.

Search maps RBAC for resources in the hub cluster. Depending on user RBAC settings for Red Hat Advanced Cluster Management, users might not see node data from the managed cluster. Results from search might be different from what is displayed on the Nodes page for a cluster.

When you destroy a managed cluster, the status continues to display Destroying after one hour, and the cluster is not destroyed. To resolve this issue complete the following steps:

You cannot use the Red Hat Advanced Cluster Management console to upgrade OpenShift Container Platform managed clusters that are in the Red Hat OpenShift Dedicated environment.

Your bare metal asset might remain as an orphaned asset after you destroy the cluster that was associated with it. This happens when you have the required permissions to destroy a cluster, but not to destroy the bare metal asset. To ensure that you do not experience this issue, add a finalizer to the ClusterDeployment resource when you create a bare metal asset with Red Hat Advanced Cluster Management that references a cluster deployment:

kubectl patch clusterdeployments <name> -n <namespace> -p '{"metadata":{"finalizers":["baremetalasset.inventory.open-cluster-management.io"]}}'

Replace name with the name of your cluster deployment.

Replace namespace with the namespace of your cluster resource.

If you delete the cluster deployment, you must remove the finalizer manually by entering the following command:

kubectl patch clusterdeployments <name> -n <namespace> -p '{"metadata":{"finalizers":[]}}'

Replace name with the name of your cluster deployment.

Replace namespace with the namespace of your cluster resource.

When you create an application with a deployment window that is set to Active within specified interval, the deployment window might not be calculated correctly, resulting in the application being deployed in undefined times.

An application name cannot exceed 37 characters. The application deployment displays the following error if the characters exceed this amount:

status:
  phase: PropagationFailed

The search details page for a certain resource on a certain managed cluster might fail. You must ensure that the work-manager add-on in the managed cluster is in Available status before you can search.

Applying a Deployable resource with no specification crashes the pod multicluster-operators-application container multicluster-operators-deployable. A deployable needs to contain specifications.

If you accidentally create the resource without a specification, delete the unnecessary deployable and restart the multicluster-operators-application pod.

See the following example of a Deployable that is empty and crashes the pod:

apiVersion: apps.open-cluster-management.io/v1
kind: Deployable
metadata:
  labels:
    app: simple-app-tester
  name: simple-app-tester-deployable
  namespace: grp-proof-of-concept-acm

When you deploy an application that directly creates a ReplicationController or ReplicaSet resource, the Pod resources are not displayed in the Application topology. You can use the Deployment or DeploymentConfig resources instead for creating Pod resources.

Ansible tests run as hooks for the subscription and not as regular tasks. You need to store Ansible tasks in a prehook and posthook folder. You can choose to deploy the Ansible tasks as regular tasks and not as hooks, but the Application topology Ansible job status is not reported correctly in this case.

Ansible hook stand-alone mode is not supported. To deploy Ansible hook on the hub cluster with a subscription, you might use the following subscription YAML:

apiVersion: apps.open-cluster-management.io/v1
kind: Subscription
metadata:
  name: sub-rhacm-gitops-demo
  namespace: hello-openshift
annotations:
  apps.open-cluster-management.io/github-path: myapp
  apps.open-cluster-management.io/github-branch: master
spec:
  hooksecretref:
      name: toweraccess
  channel: rhacm-gitops-demo/ch-rhacm-gitops-demo
  placement:
     local: true

However, this configuration might never create the Ansible instance, since the spec.placement.local:true has the subscription running on standalone mode. You need to create the subscription in hub mode.

After applying both, you should see the Ansible instance created in your hub cluster.

If you select Deploy on local cluster when you create or edit an application, the application Topology does not display correctly. Deploy on local cluster is the option to deploy resources on your hub cluster so that you can manage it as the local cluster, but this is not best practice for this release.

To resolve the issue, see the following procedure:

When you subscribe to a namespace channel and the subscription remains in FAILED state after you fixed other associated resources such as channel, secret, ConfigMap, or placement rule, the namespace subscription is not continuously reconciled.

To force the subscription reconcile again to get out of FAILED state, complete the following steps:

oc label subscriptions.apps.open-cluster-management.io the_subscription_name reconcile=true

A user performing in an Editor role should only have read or update authority on an application, but erroneously editor can also create and delete an application. Red Hat OpenShift Operator Lifecycle Manager default settings change the setting for the product. To workaround the issue, see the following procedure:

A user performing in an Editor role should only have read or update authority on an placement rule, but erroneously editor can also create and delete, as well. Red Hat OpenShift Operator Lifecycle Manager default settings change the setting for the product. To workaround the issue, see the following procedure:

If applications are not deploying after an update to a placement rule, verify that the klusterlet-addon-appmgr pod is running. The klusterlet-addon-appmgr is the subscription container that needs to run on endpoint clusters.

You can run oc get pods -n open-cluster-management-agent-addon to verify.

You can also search for kind:pod cluster:yourcluster in the console and see if the klusterlet-addon-appmgr is running.

If you cannot verify, attempt to import the cluster again and verify again.

Learn about Red Hat OpenShift Container Platform SCC at Managing Security Context Constraints (SCC), which is an additional configuration required on the managed cluster.

Different deployments have different security context and different service accounts. The subscription operator cannot create an SCC automatically. Administrators control permissions for pods. A Security Context Constraints (SCC) CR is required to enable appropriate permissions for the relative service accounts to create pods in the non-default namespace:

To manually create an SCC CR in your namespace, complete the following:

Creating more than one channel in the same namespace can cause errors with the hub cluster.

For instance, namespace charts-v1 is used by the installer as a Helm type channel, so do not create any additional channels in charts-v1. Ensure that you create your channel in a unique namespace. All channels need an individual namespace, except GitHub channels, which can share a namespace with another GitHub channel.

See the following limitations to various Application tables in the console:

When you have a configuration policy that is configured with mustnothave for the complianceType parameter and enforce for the remediationAction parameter, the policy is listed as compliant after a deletion request is made to the Kubernetes API. Therefore, the Kubernetes object can be stuck in a Terminating state while the policy is listed as compliant.

Red Hat Advanced Cluster Management follows the Kubernetes deprecation guidelines for APIs. See the Kubernetes Deprecation Policy for more details about that policy.

Red Hat Advanced Cluster Management APIs are only deprecated or removed outside of the following timelines:

GDPR establishes a stronger data protection regulatory framework for processing personal data of individuals. GDPR brings:

As a platform, Red Hat Advanced Cluster Management for Kubernetes deals with several categories of technical data that could be considered as personal data, such as an administrator user ID and password, service user IDs and passwords, IP addresses, and Kubernetes node names. The Red Hat Advanced Cluster Management for Kubernetes platform also deals with information about users who manage the platform. Applications that run on the platform might introduce other categories of personal data unknown to the platform.

Information on how this technical data is collected/created, stored, accessed, secured, logged, and deleted is described in later sections of this document.

Customers can submit online comments/feedback/requests for information about in a variety of ways, primarily:

Typically, only the client name and email address are used, to enable personal replies for the subject of the contact, and the use of personal data conforms to the Red Hat Online Privacy Statement.

The Red Hat Advanced Cluster Management for Kubernetes platform authentication manager accepts user credentials from the console and forwards the credentials to the backend OIDC provider, which validates the user credentials against the enterprise directory. The OIDC provider then returns an authentication cookie (auth-cookie) with the content of a JSON Web Token (JWT) to the authentication manager. The JWT token persists information such as the user ID and email address, in addition to group membership at the time of the authentication request. This authentication cookie is then sent back to the console. The cookie is refreshed during the session. It is valid for 12 hours after you sign out of the console or close your web browser.

For all subsequent authentication requests made from the console, the front-end NGINX server decodes the available authentication cookie in the request and validates the request by calling the authentication manager.

The Red Hat Advanced Cluster Management for Kubernetes platform CLI requires the user to provide credentials to log in.

The kubectl and oc CLI also requires credentials to access the cluster. These credentials can be obtained from the management console and expire after 12 hours. Access through service accounts is supported.

Red Hat Advanced Cluster Management for Kubernetes platform supports role-based access control (RBAC). In the role mapping stage, the user name that is provided in the authentication stage is mapped to a user or group role. The roles are used when authorizing which administrative activities can be carried out by the authenticated user.

Red Hat Advanced Cluster Management for Kubernetes platform roles control access to cluster configuration actions, to catalog and Helm resources, and to Kubernetes resources. Several IAM (Identity and Access Management) roles are provided, including Cluster Administrator, Administrator, Operator, Editor, Viewer. A role is assigned to users or user groups when you add them to a team. Team access to resources can be controlled by namespace.

Pod security policies are used to set up cluster-level control over what a pod can do or what it can access.