Chapter 1. Risk and compliance

Manage your security of Red Hat Advanced Cluster Management for Kubernetes components. Govern your cluster with defined policies and processes to identify and minimize risks. Use policies to define rules and set controls.

Prerequisite: You must configure authentication service requirements for Red Hat Advanced Cluster Management for Kubernetes. See Access control for more information.

Review the following topics to learn more about securing your cluster:

1.1. Certificates

Various certificates are created and used throughout Red Hat Advanced Cluster Management for Kubernetes.

You can bring your own certificates. You must create a Kubernetes TLS Secret for your certificate. After you create your certificates, you can replace certain certificates that are created by the Red Hat Advanced Cluster Management installer.

Required access: Cluster administrator

All certificates required by services that run on Red Hat Advanced Cluster Management are created during the installation of Red Hat Advanced Cluster Management. Certificates are created and managed by the following components:

OpenShift Service Serving Certificates
Red Hat Advanced Cluster Management webhook controllers
Kubernetes Certificates API
OpenShift default ingress

Continue reading to learn more about certificate management:

Red Hat Advanced Cluster Management hub cluster certificates

Red Hat Advanced Cluster Management component certificates

Red Hat Advanced Cluster Management managed certificates

Third-party certificates

Rotating the gatekeeper webhook certificate

Note: Users are responsible for certificate rotations and updates.

1.1.1. Red Hat Advanced Cluster Management hub cluster certificates

1.1.1.1. Observability certificates

After Red Hat Advanced Cluster Management is installed, observability certificates are created and used by the observability components, to provide mutual TLS on the traffic between the hub cluster and managed cluster. The Kubernetes secrets that are associated with the observability certificates.

The open-cluster-management-observability namespace contain the following certificates:

observability-server-ca-certs: Has the CA certificate to sign server-side certificates
observability-client-ca-certs: Has the CA certificate to sign client-side certificates
observability-server-certs: Has the server certificate used by the observability-observatorium-api deployment
observability-grafana-certs: Has the client certificate used by the observability-rbac-query-proxy deployment

The open-cluster-management-addon-observability namespace contain the following certificates on managed clusters:

observability-managed-cluster-certs: Has the same server CA certificate as observability-server-ca-certs in the hub server
observability-controller-open-cluster-management.io-observability-signer-client-cert: Has the client certificate used by the metrics-collector-deployment

The CA certificates are valid for five years and other certificates are valid for one year. All observability certificates are automatically refreshed upon expiration.

View the following list to understand the effects when certificates are automatically renewed:

Non-CA certificates are renewed automatically when the remaining valid time is no more than 73 days. After the certificate is renewed, the pods in the related deployments restart automatically to use the renewed certificates.
CA certificates are renewed automatically when the remaining valid time is no more than one year. After the certificate is renewed, the old CA is not deleted but co-exist with the renewed ones. Both old and renewed certificates are used by related deployments, and continue to work. The old CA certificates are deleted when they expire.
When a certificate is renewed, the traffic between the hub cluster and managed cluster is not interrupted.

1.1.1.2. Bring Your Own (BYO) observability certificate authority (CA) certificates

If you do not want to use the default observability CA certificates generated by Red Hat Advanced Cluster Management, you can choose to use the BYO observability CA certificates before you enable observability.

1.1.1.2.1. OpenSSL commands to generate CA certificate

Observability requires two CA certificates; one is for the server-side and the other is for the client-side.

Generate your CA RSA private keys with the following commands:

openssl genrsa -out serverCAKey.pem 2048

openssl genrsa -out clientCAKey.pem 2048

Generate the self-signed CA certificates using the private keys. Run the following commands:

openssl req -x509 -sha256 -new -nodes -key serverCAKey.pem -days 1825 -out serverCACert.pem

openssl req -x509 -sha256 -new -nodes -key clientCAKey.pem -days 1825 -out clientCACert.pem

1.1.1.2.2. Create the secrets associated with the BYO observability CA certificates

Complete the following steps to create the secrets:

Create the observability-server-ca-certs secret by using your certificate and private key. Run the following command:

oc -n open-cluster-management-observability create secret tls observability-server-ca-certs --cert ./serverCACert.pem --key ./serverCAKey.pem

Create the observability-client-ca-certs secret by using your certificate and private key. Run the following command:

oc -n open-cluster-management-observability create secret tls observability-client-ca-certs --cert ./clientCACert.pem --key ./clientCAKey.pem

1.1.1.2.3. Replacing certificates for alertmanager route

You can replace alertmanager certificates by updating the alertmanager route, if you do not want to use the OpenShift default ingress certificate. Complete the following steps:

Examine the observability certificate with the following command:
```
openssl x509  -noout -text -in ./observability.crt
```
Change the common name (CN) on the certificate to alertmanager.
Change the SAN in the csr.cnf configuration file with the hostname for your alertmanager route.

Create the two following secrets in the open-cluster-management-observability namespace. Run the following command:

oc -n open-cluster-management-observability create secret tls alertmanager-byo-ca --cert ./ca.crt --key ./ca.key

oc -n open-cluster-management-observability create secret tls alertmanager-byo-cert --cert ./ingress.crt --key ./ingress.key

1.1.2. Red Hat Advanced Cluster Management component certificates

1.1.2.1. List hub cluster managed certificates

You can view a list of hub cluster managed certificates that use OpenShift Service Serving Certificates service internally. Run the following command to list the certificates:

for ns in multicluster-engine open-cluster-management ; do echo "$ns:" ; oc get secret -n $ns -o custom-columns=Name:.metadata.name,Expiration:.metadata.annotations.service\\.beta\\.openshift\\.io/expiry | grep -v '<none>' ; echo ""; done

Note: If observability is enabled, there are additional namespaces where certificates are created.

1.1.2.2. Refresh hub cluster managed certificates

You can refresh a hub cluster managed certificate by running the delete secret command in the List hub cluster managed certificates section. When you identify the certificate that you need to refresh, delete the secret that is associated with the certificate. For example, you can delete a secret by running the following command:

oc delete secret grc-0c925-grc-secrets -n open-cluster-management

Note: After you delete the secret, a new one is created. However, you must restart pods that use the secret manually so they can begin to use the new certificate.

1.1.2.3. Refresh a Red Hat Advanced Cluster Management webhook certificate

You can refresh OpenShift Container Platform managed certificates, which are certificates that are used by Red Hat Advanced Cluster Management webhooks.

Complete the following steps to refresh Red Hat Advanced Cluster Management webhook certificate:

Delete the secret that is associated with the OpenShift Container Platform managed certificate by running the following command:
```
oc delete secret -n open-cluster-management ocm-webhook-secret
```
Note: Some services might not have a secret that needs to be deleted.
Restart the services that are associated with the OpenShift Container Platform managed certificate(s) by running the following command:
```
oc delete po -n open-cluster-management ocm-webhook-679444669c-5cg76
```
Important: There are replicas of many services; each service must be restarted.

View the following table for a summarized list of the pods that contain certificates and whether a secret needs to be deleted prior to restarting the pod:

Table 1.1. Pods that contain OpenShift Container Platform managed certificates
Service name	Namespace	Sample pod name	Secret name (if applicable)
channels-apps-open-cluster-management-webhook-svc	open-cluster-management	multicluster-operators-application-8c446664c-5lbfk	channels-apps-open-cluster-management-webhook-svc-ca
multicluster-operators-application-svc	open-cluster-management	multicluster-operators-application-8c446664c-5lbfk	multicluster-operators-application-svc-ca
cluster-manager-registration-webhook	open-cluster-management-hub	cluster-manager-registration-webhook-fb7b99c-d8wfc	registration-webhook-serving-cert
cluster-manager-work-webhook	open-cluster-management-hub	cluster-manager-work-webhook-89b8d7fc-f4pv8	work-webhook-serving-cert

1.1.3. Red Hat Advanced Cluster Management managed certificates

1.1.3.1. Channel certificates

CA certificates can be associated with Git channel that are a part of the Red Hat Advanced Cluster Management application management. See Using custom CA certificates for a secure HTTPS connection for more details.

Helm channels allow you to disable certificate validation. Helm channels where certificate validation is disabled, must be configured in development environments. Disabling certificate validation introduces security risks.

1.1.3.2. Managed cluster certificates

Certificates are used to authenticate managed clusters with the hub. Therefore, it is important to be aware of troubleshooting scenarios associated with these certificates. View Troubleshooting imported clusters offline after certificate change for more details.

The managed cluster certificates are refreshed automatically.

1.1.4. Third-party certificates

1.1.4.1. Rotating the gatekeeper webhook certificate

Complete the following steps to rotate the gatekeeper webhook certificate:

Edit the secret that contains the certificate with the following command:

oc edit secret -n openshift-gatekeeper-system gatekeeper-webhook-server-cert

Delete the following content in the data section: ca.crt, ca.key, tls.crt`, and tls.key.
Restart the gatekeeper webhook service by deleting the gatekeeper-controller-manager pods with the following command:
```
oc delete po -n openshift-gatekeeper-system -l control-plane=controller-manager
```

The gatekeeper webhook certificate is rotated.

Use the certificate policy controller to create and manage certificate policies on managed clusters. See Policy controllers to learn more about controllers. Return to the Risk and compliance page for more information.