Release notes

Uninstalling Red Hat Advanced Cluster Management from OpenShift Container Platform can cause issues if you later want to install earlier versions and then upgrade. For instance, when you uninstall Red Hat Advanced Cluster Management from OpenShift Container Platform, then install an earlier version of Red Hat Advanced Cluster Management and upgrade that version, the upgrade might fail. The upgrade fails if the StorageVersionMigration custom resources were not removed.

When you uninstall Red Hat Advanced Cluster Management, you need to manually remove the previous StorageVersionMigration before you reinstall and upgrade.

For example, if you uninstall Red Hat Advanced Cluster Management 2.10 from OpenShift Container Platform to use an earlier version of Red Hat Advanced Cluster Management, then you attempt to upgrade to 2.10 again, the upgrade fails unless you remove StorageVersionMigration resources.

When you install the infrastructure-operator, converged flow with ARM does not work. Set ALLOW_CONVERGED_FLOW to false to resolve this issue.

The agent appears in the inventory when the issue is resolved.

After you upgrade from 2.4.x to 2.5.x, and then to 2.6.x, deprecated resources in the managed cluster namespace might remain. You need to manually delete these deprecated resources if version 2.6.x was upgraded from 2.4.x:

Note: You need to wait 30 minutes or more before you upgrade from version 2.5.x to version 2.6.x.

You can delete from the console, or you can run a command similar to the following example for the resources you want to delete:

oc delete -n <managed cluster namespace> managedclusteraddons.addon.open-cluster-management.io <resource-name>

See the list of deprecated resources that might remain:

managedclusteraddons.addon.open-cluster-management.io:
policy-controller
manifestworks.work.open-cluster-management.io:
-klusterlet-addon-appmgr
-klusterlet-addon-certpolicyctrl
-klusterlet-addon-crds
-klusterlet-addon-iampolicyctrl
-klusterlet-addon-operator
-klusterlet-addon-policyctrl
-klusterlet-addon-workmgr

After upgrading Red Hat Advanced Cluster Management to a new version, a few pods that belong to a StatefulSet might remain in a failed state. This infrequent event is caused by a known Kubernetes issue.

As a workaround for this problem, delete the failed pod. Kubernetes automatically relaunches it with the correct settings.

When an OpenShift Container Platform cluster is in the upgrade stage, the cluster pods are restarted and the cluster might remain in upgrade failed status for a variation of 1-5 minutes. This behavior is expected and resolves after a few minutes.

After installing Red Hat Advanced Cluster Management for Kubernetes in the Red Hat OpenShift Container Platform console, a pop-up window with the following message appears:

MultiClusterEngine required

Create a MultiClusterEngine instance to use this Operator.

The Create MultiClusterEngine button in the pop-up window message might not work. To work around the issue, select Create instance in the MultiClusterEngine tile in the Provided APIs section.

Backup and restore known issues and limitations are listed here, along with workarounds if they are available.

E0430 19:11:20.810624 1 clientset.go:188] "cannot connect once" err="rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\""

oc get backupschedule -n open-cluster-management-backup
NAME PHASE MESSAGE
rosa-backup-schedule FailedValidation Backup storage location is not available. Check velero.io.BackupStorageLocation and validate storage credentials.

spec:
  clusterReplicas: 1
  clusterSelector:
    matchLabels:
      environment: dev

spec:
  clusterSelector:
    matchLabels:
      environment: dev

oc delete appliedmanifestwork <the-left-appliedmanifestwork-name>

From the console you can request an upgrade for OpenShift Dedicated clusters, but the upgrade fails with the Cannot upgrade non openshift cluster error message. Currently there is no workaround.

The search-postgres pod is in CrashLoopBackoff state. If Red Hat Advanced Cluster Management is deployed in a cluster with nodes that have the hugepages parameter enabled and the search-postgres pod gets scheduled in these nodes, then the pod does not start.

Complete the following steps to increase the memory of the search-postgres pod:

The following value appears, 512Mi.

When you edit namespace bindings for a cluster set with the admin role or bind role, you might encounter an error that resembles the following message:

ResourceError: managedclustersetbindings.cluster.open-cluster-management.io "<cluster-set>" is forbidden: User "<user>" cannot create/delete resource "managedclustersetbindings" in API group "cluster.open-cluster-management.io" in the namespace "<namespace>".

To resolve the issue, make sure you also have permission to create or delete a ManagedClusterSetBinding resource in the namespace you want to bind. The role bindings only allow you to bind the cluster set to the namespace.

After provisioning a hosted control plane cluster, you might not be able to scroll horizontally in the cluster overview of the Red Hat Advanced Cluster Management console if the ClusterVersionUpgradeable parameter is too long. You cannot view the hidden data as a result.

To work around the issue, zoom out by using your browser zoom controls, increase your Red Hat Advanced Cluster Management console window size, or copy and paste the text to a different location.

When you add multiple label expressions or attempt to enter your cluster selector for your ApplicationSet, you might receive the following message repeatedly, "Expand to enter expression". You can enter your cluster selection despite this issue.

If you have OpenShift Container Platform 4.17, the search serviceaccount is missing some permissions it needs to function as a user. Therefore, the Argo CD pull model resource sync controller fails to access the application resource list from the search API.

To work around this issue, complete the following steps:

Note: Red Hat Advanced Cluster Management support of (OpenShift Container Platform 3.11 is deprecated.

After you create a subscription application that targets a OpenShift Container Platform 3.11 cluster, the application topology displays incorrectly for ReplicaSet and Pod resources because of a defect in Kubernetes. This defect is where the pod-template-hash does not match the hash in the ReplicaSet or Pod resource name. Later Kubernetes version are corrected, but OpenShift Container Platform 3.11 is not fixed. See Kubernetes bug reference for details.

Because of this bug, the topology might not reflect the status of resources. For example, pod and replicaset are not reflected, but those resources exist.

The application add-on component uses the Kubernetes Lease API ,leases.coordination.k8s.io, which is missing for OpenShift Container Platform 3.11 users. The Kubernetes Lease API was introduced in Kubernetes 1.14, but OpenShift Container Platform 3.11 bundles Kubernetes version 1.11.

To resolve this issue, manually apply the following Kubernetes Lease API CustomResourceDefinition to the OpenShift Container Platform 3.11 managed cluster:

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: leases.coordination.k8s.io
spec:
  group: coordination.k8s.io
  names:
    kind: Lease
    listKind: LeaseList
    plural: leases
    singular: lease
    shortNames:
    - ls
  scope: Namespaced
  versions:
  - name: v1
    served: true    storage: true    schema:
      openAPIV3Schema:
        description: Lease defines a lease concept.
        type: object
        properties:
          apiVersion:
            type: string
          kind:
            type: string
          metadata:
            type: object
          spec:
            type: object
            properties:
              acquireTime:
                format: date-time
                type: string
              holderIdentity:
                type: string
              leaseDurationSeconds:
                format: int64
                type: integer
              leaseTransitions:
                format: int64
                type: integer
              renewTime:
                format: date-time
                type: string
            required:
            - holderIdentity
            - leaseDurationSeconds
            - renewTime
        required:
        - kind
        - metadata
        - spec
  additionalPrinterColumns:
  - JSONPath: .metadata.creationTimestamp
    name: Age
    type: date
  subresources:
    status: {}

Note: Red Hat Advanced Cluster Management support of (OpenShift Container Platform 3.11 is deprecated.

When you create a service account in Red Hat OpenShift Container Platform 4.15 that is provisioned by some cloud providers, such as IBM VMware and Bare Metal, then the account does not automatically create a secret. Therefore, the Red Hat Advanced Cluster Management gitopsCluster controller fails to generate the managed cluster secret for the Argo CD push model.

This issue does not occur in Red Hat OpenShift Container Platform 4.15 that is provisioned by AWS. However, the issue can occur in Red Hat OpenShift Container Platform 4.15 provisioned by other cloud providers. This issue is delivered in Red Hat Advanced Cluster Management 2.10.3 and in Red Hat Advanced Cluster Management 2.9.4.

To workaround this issue, you must manually create a secret and attach it to your service account, open-cluster-management-agent-addon/application-manager. To do this, complete the following steps:

To verify that you successfully created a secret and attached it to your service account, complete the following steps:

After you create a subscription application that references a PlacementRule resource, the subscription YAML does not display in the YAML editor in the console. Use your terminal to edit your subscription YAML file.

Using Helm Chart, you can define privacy data in a Kubernetes secret and refer to this secret within the value.yaml file of the Helm Chart.

The username and password are given by the referred Kubernetes secret resource dbsecret. For example, see the following sample value.yaml file:

credentials:
  secretName: dbsecret
  usernameSecretKey: username
  passwordSecretKey: password

The Helm Chart with secret dependencies is only supported in the Helm binary CLI. It is not supported in the operator SDK Helm library. The Red Hat Advanced Cluster Management subscription controller applies the operator SDK Helm library to install and upgrade the Helm Chart. Therefore, the Red Hat Advanced Cluster Management subscription cannot deploy the Helm Chart with secret dependencies.

Customized cluster secrets cannot be created for the Argo CD Push model on your OpenShift Container Platform 3.11 managed clusters. This occurs because the managed service account add-on is not supported on OpenShift Container Platform 3.11 managed clusters.

When you use the Argo CD pull model to deploy ApplicationSet applications and the application resource names are customized, the resource names might appear different for each cluster. When this happens, the topology does not display your application correctly.

The hub cluster application set deploys to target managed clusters, but the local cluster, which is a managed hub cluster, is excluded as a target managed cluster.

As a result, if the Argo CD application is propagated to the local cluster by the Argo CD pull model, the local cluster Argo CD application is not cleaned up, even though the local cluster is removed from the placement decision of the Argo CD ApplicationSet resource.

To work around the issue and clean up the local cluster Argo CD application, remove the skip-reconcile annotation from the local cluster Argo CD application. See the following annotation:

annotations:
    argocd.argoproj.io/skip-reconcile: "true"

Additionally, if you manually refresh the pull model Argo CD application in the Applications section of the Argo CD console, the refresh is not processed and the REFRESH button in the Argo CD console is disabled.

To work around the issue, remove the refresh annotation from the Argo CD application. See the following annotation:

annotations:
    argocd.argoproj.io/refresh: normal

Both the Argo CD controller and the propagation controller might reconcile on the same application resource and cause the duplicate instances of application deployment on the managed clusters, but from the different deployment models.

For deploying applications by using the pull model, the Argo CD controllers ignore these application resources when the Argo CD argocd.argoproj.io/skip-reconcile annotation is added to the template section of the ApplicationSet.

The argocd.argoproj.io/skip-reconcile annotation is only available in the GitOps operator version 1.9.0, or later. To prevent conflicts, wait until the hub cluster and all the managed clusters are upgraded to GitOps operator version 1.9.0 before implementing the pull model.

All the resources listed in the MulticlusterApplicationSetReport are actually deployed on the managed clusters. If a resource fails to deploy, the resource is not included in the resource list, but the cause is listed in the error message.

For large environments with over 1000 managed clusters and Argo CD application sets that are deployed to hundreds of managed clusters, Argo CD application creation on the hub cluster might take several minutes. You can set the requeueAfterSeconds to zero in the clusterDecisionResource generator of the application set, as it is displayed in the following example file:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cm-allclusters-app-set
  namespace: openshift-gitops
spec:
  generators:
  - clusterDecisionResource:
      configMapRef: ocm-placement-generator
      labelSelector:
        matchLabels:
          cluster.open-cluster-management.io/placement: app-placement
      requeueAfterSeconds: 0

You cannot specify allow and deny lists with ObjectBucket channel type in the subscription-admin role. In other channel types, the allow and deny lists in the subscription indicates which Kubernetes resources can be deployed, and which Kubernetes resources should not be deployed.

The application-manager add-on that is running on the managed clusters is now handled by the subscription operator, when it was previously handled by the klusterlet operator. The subscription operator is not managed the multicluster-hub, so changes to the multicluster_operators_subscription image in the multicluster-hub image manifest ConfigMap do not take effect automatically.

If the image that is used by the subscription operator is overrided by changing the multicluster_operators_subscription image in the multicluster-hub image manifest ConfigMap, the application-manager add-on on the managed clusters does not use the new image until the subscription operator pod is restarted. You need to restart the pod.

The policy.open-cluster-management.io/v1 resources are no longer deployed by an application subscription by default for Red Hat Advanced Cluster Management version 2.4.

A subscription administrator needs to deploy the application subscription to change this default behavior.

See Creating an allow and deny list as subscription administrator for information. policy.open-cluster-management.io/v1 resources that were deployed by existing application subscriptions in previous Red Hat Advanced Cluster Management versions remain, but are no longer reconciled with the source repository unless the application subscriptions are deployed by a subscription administrator.

Ansible hook stand-alone mode is not supported. To deploy Ansible hook on the hub cluster with a subscription, you might use the following subscription YAML:

apiVersion: apps.open-cluster-management.io/v1
kind: Subscription
metadata:
  name: sub-rhacm-gitops-demo
  namespace: hello-openshift
annotations:
  apps.open-cluster-management.io/github-path: myapp
  apps.open-cluster-management.io/github-branch: master
spec:
  hooksecretref:
      name: toweraccess
  channel: rhacm-gitops-demo/ch-rhacm-gitops-demo
  placement:
     local: true

However, this configuration might never create the Ansible instance, since the spec.placement.local:true has the subscription running on standalone mode. You need to create the subscription in hub mode.

After applying both, you should see the Ansible instance created in your hub cluster.

If applications are not deploying after an update to a placement rule, verify that the application-manager pod is running. The application-manager is the subscription container that needs to run on managed clusters.

You can run oc get pods -n open-cluster-management-agent-addon |grep application-manager to verify.

You can also search for kind:pod cluster:yourcluster in the console and see if the application-manager is running.

If you cannot verify, attempt to import the cluster again and verify again.

Learn about Red Hat OpenShift Container Platform SCC at Managing Security Context Constraints (SCC), which is an additional configuration required on the managed cluster.

Different deployments have different security context and different service accounts. The subscription operator cannot create an SCC CR automatically.. Administrators control permissions for pods. A Security Context Constraints (SCC) CR is required to enable appropriate permissions for the relative service accounts to create pods in the non-default namespace. To manually create an SCC CR in your namespace, complete the following steps:

Creating more than one channel in the same namespace can cause errors with the hub cluster.

For instance, namespace charts-v1 is used by the installer as a Helm type channel, so do not create any additional channels in charts-v1. Ensure that you create your channel in a unique namespace. All channels need an individual namespace, except GitHub channels, which can share a namespace with another GitHub channel.

Ansible jobs fail to run when you select an incompatible option. Ansible Automation Platform only works when the -cluster-scoped channel options are chosen. This affects all components that need to perform Ansible jobs.

The Red Hat Ansible Automation Platform operator cannot access Ansible Automation Platform outside of a proxy-enabled OpenShift Container Platform cluster. To resolve, you can install the Ansible Automation Platform within the proxy. See install steps that are provided by Ansible Automation Platform.

An application name cannot exceed 37 characters. The application deployment displays the following error if the characters exceed this amount.

status:
  phase: PropagationFailed
  reason: 'Deployable.apps.open-cluster-management.io "_long_lengthy_name_" is invalid: metadata.labels: Invalid value: "_long_lengthy_name_": must be no more than 63 characters/n'

See the following limitations to various Application tables in the console:

The Console and Topology for Application changes for the 2.10. There is no filtering capability from the console Topology page.

The allow and deny list feature does not work in Object storage application subscriptions.

The Observatorium API gateway pods in a restored hub cluster might contain stale tenant data after a backup and restore procedure because of a Kubernetes limitation. See Mounted ConfigMaps are updated automatically for more about the limitation.

As a result, the Observatorium API and Thanos gateway rejects metrics from collectors, and the Red Hat Advanced Cluster Management Grafana dashboards do not display data.

See the following errors from the Observatorium API gateway pod logs:

level=error name=observatorium caller=logchannel.go:129 msg="failed to forward metrics" returncode="500 Internal Server Error" response="no matching hashring to handle tenant\n"

Thanos receives pods logs with the following errors:

caller=handler.go:551 level=error component=receive component=receive-handler tenant=xxxx err="no matching hashring to handle tenant" msg="internal server error"

See the following procedure to resolve this issue:

Note: N = 2 by default, but might be greater than 2 in some custom configuration environments.

This restarts all Observatorium API gateway pods with the correct tenant information, and the data from collectors start displaying in Grafana in between 5-10 minutes.

Starting with Red Hat Advanced Cluster Management 2.9, you must use a label in your defined Red Hat Advanced Cluster Management hub cluster namespace. The label, openshift.io/cluster-monitoring: "true" causes the Cluster Monitoring Operator to scrape the namespace for metrics.

When Red Hat Advanced Cluster Management 2.9 is deployed or an installation is upgraded to 2.9, the Red Hat Advanced Cluster Management Observability ServiceMonitors and PrometheusRule resources are no longer present in the openshift-monitoring namespace.

The Prometheus AdditionalAlertManagerConfig resource of the observability add-on does not support proxy settings. You must disable the observability alert forwarding feature.

Complete the following steps to disable alert forwarding:

The HTTPS proxy with a self-signed CA certificate is not supported.

When various hub clusters deploy Red Hat Advanced Cluster Management observability using the same S3 storage, duplicate local-clusters can be detected and displayed within the Kubernetes/Service-Level Overview/API Server dashboard. The duplicate clusters affect the results within the following panels: Top Clusters, Number of clusters that has exceeded the SLO, and Number of clusters that are meeting the SLO. The local-clusters are unique clusters associated with the shared S3 storage. To prevent multiple local-clusters from displaying within the dashboard, it is recommended for each unique hub cluster to deploy observability with a S3 bucket specifically for the hub cluster.

The observability endpoint operator fails if you create a pull-secret to deploy to the MultiClusterObservability CustomResource (CR) and there is no pull-secret in the open-cluster-management-observability namespace. When you import a new cluster, or import a Hive cluster that is created with Red Hat Advanced Cluster Management, you need to manually create a pull-image secret on the managed cluster.

For more information, see Enabling observability.

Red Hat Advanced Cluster Management observability does not display data from a ROKS cluster on some panels within built-in dashboards. This is because ROKS does not expose any API server metrics from servers they manage. The following Grafana dashboards contain panels that do not support ROKS clusters: Kubernetes/API server, Kubernetes/Compute Resources/Workload, Kubernetes/Compute Resources/Namespace(Workload)

For ROKS clusters, Red Hat Advanced Cluster Management observability does not display data in the etcd panel of the dashboard.

By default, Prometheus on OpenShift uses ephemeral storage. Prometheus loses all metrics data whenever it is restarted.

When observability is enabled or disabled on OpenShift Container Platform managed clusters that are managed by Red Hat Advanced Cluster Management, the observability endpoint operator updates the cluster-monitoring-config ConfigMap by adding additional alertmanager configuration that restarts the local Prometheus automatically.

Observability receive pods report the following error message:

Error on ingesting out-of-order samples

The error message means that the time series data sent by a managed cluster, during a metrics collection interval is older than the time series data it sent in the previous collection interval. When this problem happens, data is discarded by the Thanos receivers and this might create a gap in the data shown in Grafana dashboards. If the error is seen frequently, it is recommended to increase the metrics collection interval to a higher value. For example, you can increase the interval to 60 seconds.

The problem is only noticed when the time series interval is set to a lower value, such as 30 seconds. Note, this problem is not seen when the metrics collection interval is set to the default value of 300 seconds.

If you have a grafana-dev instance deployed in earlier versions before 2.6, and you upgrade the environment to 2.6, the grafana-dev does not work. You must delete the existing grafana-dev instance by running the following command:

./setup-grafana-dev.sh --clean

Recreate the instance with the following command:

./setup-grafana-dev.sh --deploy

The klusterlet-addon-search pod fails because the memory limit is reached. You must update the memory request and limit by customizing the klusterlet-addon-search deployment on your managed cluster. Edit the ManagedclusterAddon custom resource named search-collector, on your hub cluster. Add the following annotations to the search-collector and update the memory, addon.open-cluster-management.io/search_memory_request=512Mi and addon.open-cluster-management.io/search_memory_limit=1024Mi.

For example, if you have a managed cluster named foobar, run the following command to change the memory request to 512Mi and the memory limit to 1024Mi:

oc annotate managedclusteraddon search-collector -n foobar \
addon.open-cluster-management.io/search_memory_request=512Mi \
addon.open-cluster-management.io/search_memory_limit=1024Mi

The Grafana dashboard shows an empty label list if the disableHubSelfManagement parameter is set to true in the mulitclusterengine custom resource. You must set the parameter to false or remove the parameter to see the label list. See disableHubSelfManagement for more details.

Endpoint url cannot have fully qualified paths

endpoint: example.com:443

A proxy configuration in a managed cluster that you configure by using the addonDeploymentConfig is not detected by the metrics collector. As a workaround, you can enable the proxy by removing the managed cluster ManifestWork. Removing the ManifestWork forces the changes in the addonDeploymentConfig to be applied.

A proxy configuration in a managed cluster does not work when a custom CA bundle is required.

The container security operator is not available in OpenShift Container Platform 3.11. Therefore, you cannot take the policy template of policy-imagemanifestvuln-sub in the ImageManifestVuln policy and apply it for OpenShift Container Platform 3.11 clusters.

If you try to apply the ImageManifestVuln policy, you receive the following violation message:

violation - couldn't find mapping resource with kind Subscription, please check if you have CRD deployed.

Governance resources are not cleaned up properly. When the component is set to false or is disabled in the MultiClusterHub operator, the governance component is removed before it can clean up the add-ons that it manages.

When you use an external identity provider to log in to Red Hat Advanced Cluster Management, you might not be able to log out of Red Hat Advanced Cluster Management. This occurs when you use Red Hat Advanced Cluster Management, installed with IBM Cloud and Keycloak as the identity providers.

You must log out of the external identity provider before you attempt to log out of Red Hat Advanced Cluster Management.

When you have a configuration policy that is configured with mustnothave for the complianceType parameter and enforce for the remediationAction parameter, the policy is listed as compliant when a deletion request is made to the Kubernetes API. Therefore, the Kubernetes object can be stuck in a Terminating state while the policy is listed as compliant.

While installation into an ARM environment is supported, operators that are deployed with policies might not support ARM environments. The following policies that install operators do not support ARM environments:

When you remove the config-policy-controller add-on from a managed cluster by disabling the policy controller in the KlusterletAddonConfig or by detaching the cluster, the ConfigurationPolicy custom resource definition might get stuck in a terminating state. If the ConfigurationPolicy custom resource definition is stuck in a terminating state, new policies might not be added to the cluster if the add-on is reinstalled later. You can also receive the following error:

template-error; Failed to create policy template: create not allowed while custom resource definition is terminating

Use the following command to check if the custom resource definition is stuck:

oc get crd configurationpolicies.policy.open-cluster-management.io -o=jsonpath='{.metadata.deletionTimestamp}'

If a deletion timestamp is on the resource, the custom resource definition is stuck. To resolve the issue, remove all finalizers from configuration policies that remain on the cluster. Use the following command on the managed cluster and replace <cluster-namespace> with the managed cluster namespace:

oc get configurationpolicy -n <cluster-namespace> -o name | xargs oc patch -n <cluster-namespace> --type=merge -p '{"metadata":{"finalizers": []}}'

The configuration policy resources are automatically removed from the cluster and the custom resource definition exits its terminating state. If the add-on has already been reinstalled, the custom resource definition is recreated automatically without a deletion timestamp.

When you modify an existing configuration policy, pruneObjectBehavior does not work. View the following reasons why pruneObjectBehavior might not work:

If a policy is set to remediationAction: enforce and is repeatedly updated, the Red Hat Advanced Cluster Management console shows repeated violations with successful updates. See the following two possible causes and solutions for the error:

The support of pod security policies is removed from OpenShift Container Platform 4.12 and later, and from Kubernetes v1.25 and later. If you apply a PodSecurityPolicy resource, you might receive the following non-compliant message:

violation - couldn't find mapping resource with kind PodSecurityPolicy, please check if you have CRD deployed

When you create a policy with identical policy template names, you receive inconsistent results that are not detected, but you might not know the cause. For example, defining a policy with multiple configuration policies named create-pod causes inconsistent results. Best practice: Avoid using duplicate names for policy templates.

When you disable governance deployments in the MultiClusterHub object, the deployments are not cleaned without errors. Complete the following steps to disable governance so that the deployments also get cleaned up:

There is built-in resilience for database and policy compliance history API outages, however, any compliance events that cannot be recorded by a managed cluster are queued in memory until they are successfully recorded. This means that if there is an outage and the governance-policy-framework pod on the managed cluster restarts, all queued compliance events are lost.

If you create or update a new policy during a database outage, any compliance events sent for this new policy cannot be recorded since the mapping of policies to database IDs cannot be updated. When the database is back online, the mapping is automatically updated and future compliance events from those policies are recorded.

If there is data loss to the PostgreSQL server such as restoring to a backup without the latest data, the governance policy propagator on the Red Hat Advanced Cluster Management hub cluster must be restarted so that it can update the mapping of policies to database IDs. Until you restart the governance policy propagator, new compliance events associated with policies that once existed in the database are no longer recorded.

To restart the governance policy propagator, run the following command on the Red Hat Advanced Cluster Management hub cluster:

oc -n open-cluster-management rollout restart deployment/grc-policy-propagator

See the following known issues and limitations that might occur while using networking features.

graceful termination failed, controllers failed with error: the server could not find the requested resource (post clustermanagementaddons.addon.open-cluster-management.io)

apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
kind: SubmarinerConfig
metadata:
   name: submariner
   namespace: <managed-cluster-namespace>
spec:
   insecureBrokerConnection: true

There are three topics named spec, status and event in your own Kafka that you brought. After upgrading multicluster global hub to 1.3 or later, only the spec and status topics remain. Configure these topics through the multicluster global hub operand. The default topis for your transport that your brought are gh-spec and gh-status.

The multicluster global hub operand is the API or resource of your system, and the multicluster global hub operator is the controller. After upgrading multicluster global hub, align the topics between your own Kafka clusters and the multicluster global hub operand. You can either create the gh-spec and gh-status topic in your Kafka cluster, or you can change the multicluster global hub operand to use the existing spec and status topics.

Use the following YAML sample to update your multicluster global hub operand to use your own Kafka topics:

apiVersion: operator.open-cluster-management.io/v1alpha4
kind: MulticlusterGlobalHub
spec:
  availabilityConfig: Basic
  dataLayer:
    kafka:
      topics:
        specTopic: spec
        statusTopic: status
    postgres:
      retention: 18m
  enableMetrics: false

If multicluster global hub is running on the latest OpenShift Container Platform environment that is FIPS-enabled, you cannot access the Grafana console due to the wrong oauth-proxy image. Red Hat Advanced Cluster Management 2.10.x supports the latest OpenShift Container Platform version, enabling you to get the oauth-proxy image from Red Hat Advanced Cluster Management 2.10.x.

To access your Grafana console, manually update the oauth-proxy image with the Red Hat Advanced Cluster Management bundle image. To update the oauth-proxy image, complete the following steps:

In the Federal Information Processing Standard (FIPS) environment, the Kafka operator keeps restarting because of the out-of-memory (OOM) state. To fix this issue, set the resource limit to at least 512M. For detailed steps on how to set this limit, see amq stream doc.

If your original multicluster global hub cluster crashes, the multicluster global hub loses its generated events and cron jobs. Even if you restore the new multicluster global hub cluster, the events and cron jobs are not restored. To workaround this issue, you can manually run the cron job, see Running the summarization process manually.

A managed cluster that is not created successfully, meaning clusterclaim id.k8s.io does not exist in the managed cluster, is not counted in the policy compliance dashboards, but shows in the policy console.

If the multicluster global hub Operator is installed on OpenShift Container Platform 4.13, all hyperlinks that link to the managed clusters list and detail pages in dashboards might redirect to the Red Hat Advanced Cluster Management home page.

You need to manually go to your target page.

In the Global Hub Policy Group Compliancy Overview hub dashboards, you can check one data point by clicking View Offending Policies for standard group, but after you click this link to go to the offending page, the standard group filter cannot pass to the new page.

This is also an issue for the Cluster Group Compliancy Overview.

If a managed hub cluster imports an OpenShift Container Platform 3.11 cluster (deprecated) as managed cluster, it cannot redirect to the Observability page in the Global Hub > Overview dashboard.

You need to manually navigate to your target page.

GDPR establishes a stronger data protection regulatory framework for processing personal data of individuals. GDPR brings:

As a platform, Red Hat Advanced Cluster Management for Kubernetes deals with several categories of technical data that could be considered as personal data, such as an administrator user ID and password, service user IDs and passwords, IP addresses, and Kubernetes node names. The Red Hat Advanced Cluster Management for Kubernetes platform also deals with information about users who manage the platform. Applications that run on the platform might introduce other categories of personal data unknown to the platform.

Information on how this technical data is collected/created, stored, accessed, secured, logged, and deleted is described in later sections of this document.

Customers can submit online comments/feedback/requests for information about in a variety of ways, primarily:

Typically, only the client name and email address are used, to enable personal replies for the subject of the contact, and the use of personal data conforms to the Red Hat Online Privacy Statement.

The Red Hat Advanced Cluster Management for Kubernetes platform authentication manager accepts user credentials from the console and forwards the credentials to the backend OIDC provider, which validates the user credentials against the enterprise directory. The OIDC provider then returns an authentication cookie (auth-cookie) with the content of a JSON Web Token (JWT) to the authentication manager. The JWT token persists information such as the user ID and email address, in addition to group membership at the time of the authentication request. This authentication cookie is then sent back to the console. The cookie is refreshed during the session. It is valid for 12 hours after you sign out of the console or close your web browser.

For all subsequent authentication requests made from the console, the front-end NGINX server decodes the available authentication cookie in the request and validates the request by calling the authentication manager.

The Red Hat Advanced Cluster Management for Kubernetes platform CLI requires the user to provide credentials to log in.

The kubectl and oc CLI also requires credentials to access the cluster. These credentials can be obtained from the management console and expire after 12 hours. Access through service accounts is supported.

Red Hat Advanced Cluster Management for Kubernetes platform supports role-based access control (RBAC). In the role mapping stage, the user name that is provided in the authentication stage is mapped to a user or group role. The roles are used when authorizing which administrative activities can be carried out by the authenticated user.

Red Hat Advanced Cluster Management for Kubernetes platform roles control access to cluster configuration actions, to catalog and Helm resources, and to Kubernetes resources. Several IAM (Identity and Access Management) roles are provided, including Cluster Administrator, Administrator, Operator, Editor, Viewer. A role is assigned to users or user groups when you add them to a team. Team access to resources can be controlled by namespace.

Pod security policies are used to set up cluster-level control over what a pod can do or what it can access.