Chapter 1. Security overview
Manage the security of your Red Hat Advanced Cluster Management for Kubernetes components. Govern your cluster with defined policies and processes to identify and minimize risks. Use policies to define rules and set controls.
Prerequisite: You must configure authentication service requirements for Red Hat Advanced Cluster Management for Kubernetes. See Access control for more information.
Read through the following topics to learn more about securing your cluster:
1.1. Certificates introduction
You can use various certificates to verify authenticity for your Red Hat Advanced Cluster Management for Kubernetes cluster. Additionally, you can bring your own certificates. Continue reading to learn about certificate management.
1.2. Certificates
All certificates required by services that run on Red Hat Advanced Cluster Management are created during the installation of Red Hat Advanced Cluster Management. View the following list of certificates, which are created and managed by the following components of Red Hat OpenShift Container Platform:
- OpenShift Service Serving Certificates
- Red Hat Advanced Cluster Management webhook controllers
- Kubernetes Certificates API
- OpenShift default ingress
Required access: Cluster administrator
Continue reading to learn more about certificate management:
Note: Users are responsible for certificate rotations and updates.
1.2.1. Red Hat Advanced Cluster Management hub cluster certificates
OpenShift default ingress certificate is technically a hub cluster certificate. After the Red Hat Advanced Cluster Management installation, observability certificates are created and used by the observability components to provide mutual TLS on the traffic between the hub cluster and managed cluster.
The
open-cluster-management-observability
namespace contains the following certificates:-
observability-server-ca-certs
: Has the CA certificate to sign server-side certificates -
observability-client-ca-certs
: Has the CA certificate to sign client-side certificates -
observability-server-certs
: Has the server certificate used by theobservability-observatorium-api
deployment -
observability-grafana-certs
: Has the client certificate used by theobservability-rbac-query-proxy
deployment
-
The
open-cluster-management-addon-observability
namespace contain the following certificates on managed clusters:-
observability-managed-cluster-certs
: Has the same server CA certificate asobservability-server-ca-certs
in the hub server -
observability-controller-open-cluster-management.io-observability-signer-client-cert
: Has the client certificate used by themetrics-collector-deployment
-
The CA certificates are valid for five years and other certificates are valid for one year. All observability certificates are automatically refreshed upon expiration. View the following list to understand the effects when certificates are automatically renewed:
- Non-CA certificates are renewed automatically when the remaining valid time is no more than 73 days. After the certificate is renewed, the pods in the related deployments restart automatically to use the renewed certificates.
- CA certificates are renewed automatically when the remaining valid time is no more than one year. After the certificate is renewed, the old CA is not deleted but co-exist with the renewed ones. Both old and renewed certificates are used by related deployments, and continue to work. The old CA certificates are deleted when they expire.
- When a certificate is renewed, the traffic between the hub cluster and managed cluster is not interrupted.
View the following Red Hat Advanced Cluster Management hub cluster certificates table:
Namespace | Secret name | Pod label | |
---|---|---|---|
open-cluster-management | channels-apps-open-cluster-management-webhook-svc-ca | app=multicluster-operators-channel | open-cluster-management |
channels-apps-open-cluster-management-webhook-svc-signed-ca | app=multicluster-operators-channel | open-cluster-management | multicluster-operators-application-svc-ca |
app=multicluster-operators-application | open-cluster-management | multicluster-operators-application-svc-signed-ca | app=multicluster-operators-application |
open-cluster-management-hub | registration-webhook-serving-cert signer-secret | Not required | open-cluster-management-hub |
1.2.2. Red Hat Advanced Cluster Management managed certificates
View the following table for a summarized list of the component pods that contain Red Hat Advanced Cluster Management managed certificates and the related secrets:
Namespace | Secret name (if applicable) |
---|---|
open-cluster-management-agent-addon | cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert |
open-cluster-management-agent-addon | cluster-proxy-service-proxy-server-certificates |
1.2.2.1. Managed cluster certificates
You can use certificates to authenticate managed clusters with the hub cluster. Therefore, it is important to be aware of troubleshooting scenarios associated with these certificates.
The managed cluster certificates are refreshed automatically.
1.2.3. Additional resources
- Use the certificate policy controller to create and manage certificate policies on managed clusters. See Certificate policy controller for more details.
- See Using custom CA certificates for a secure HTTPS connection for more details about securely connecting to a privately-hosted Git server with SSL/TLS certificates.
- See OpenShift Service Serving Certificates for more details.
- The OpenShift Container Platform default ingress is a hub cluster certificate. See Replacing the default ingress certificate for more details.
- See Certificates introduction for topics.
1.2.4. Bringing your own observability Certificate Authority (CA) certificates
When you install Red Hat Advanced Cluster Management for Kubernetes, only Certificate Authority (CA) certificates for observability are provided by default. If you do not want to use the default observability CA certificates generated by Red Hat Advanced Cluster Management, you can choose to bring your own observability CA certificates before you enable observability.
1.2.4.1. Generating CA certificates by using OpenSSL commands
Observability requires two CA certificates, one for the server-side and the other is for the client-side.
Generate your CA RSA private keys with the following commands:
openssl genrsa -out serverCAKey.pem 2048 openssl genrsa -out clientCAKey.pem 2048
Generate the self-signed CA certificates using the private keys. Run the following commands:
openssl req -x509 -sha256 -new -nodes -key serverCAKey.pem -days 1825 -out serverCACert.pem openssl req -x509 -sha256 -new -nodes -key clientCAKey.pem -days 1825 -out clientCACert.pem
1.2.4.2. Creating the secrets associated with the BYO observability CA certificates
Complete the following steps to create the secrets:
Create the
observability-server-ca-certs
secret by using your certificate and private key. Run the following command:oc -n open-cluster-management-observability create secret tls observability-server-ca-certs --cert ./serverCACert.pem --key ./serverCAKey.pem
Create the
observability-client-ca-certs
secret by using your certificate and private key. Run the following command:oc -n open-cluster-management-observability create secret tls observability-client-ca-certs --cert ./clientCACert.pem --key ./clientCAKey.pem
1.2.4.3. Additional resources
- See Managing certificates.
- Return to the Certificates introduction.
1.2.5. Managing certificates
Continue reading for information about how to refresh, replace, rotate, and list certificates.
1.2.5.1. Refreshing a Red Hat Advanced Cluster Management webhook certificate
You can refresh Red Hat Advanced Cluster Management managed certificates, which are certificates that are created and managed by Red Hat Advanced Cluster Management services.
Complete the following steps to refresh certificates managed by Red Hat Advanced Cluster Management:
Delete the secret that is associated with the Red Hat Advanced Cluster Management managed certificate by running the following command:
oc delete secret -n <namespace> <secret> 1
- 1
- Replace
<namespace>
and<secret>
with the values that you want to use.
Restart the services that are associated with the Red Hat Advanced Cluster Management managed certificate(s) by running the following command:
oc delete pod -n <namespace> -l <pod-label> 1
- 1
- Replace
<namespace>
and<pod-label>
with the values from the Red Hat Advanced Cluster Management managed cluster certificates table.
Note: If a
pod-label
is not specified, there is no service that must be restarted. The secret is recreated and used automatically.
1.2.5.2. Replacing certificates for alertmanager route
You can replace alertmanager certificates by updating the alertmanager route, if you do not want to use the OpenShift default ingress certificate. Complete the following steps:
Examine the observability certificate with the following command:
openssl x509 -noout -text -in ./observability.crt
-
Change the common name (
CN
) on the certificate toalertmanager
. -
Change the SAN in the
csr.cnf
configuration file with the hostname for your alertmanager route. Create the two following secrets in the
open-cluster-management-observability
namespace. Run the following commands:oc -n open-cluster-management-observability create secret tls alertmanager-byo-ca --cert ./ca.crt --key ./ca.key oc -n open-cluster-management-observability create secret tls alertmanager-byo-cert --cert ./ingress.crt --key ./ingress.key
1.2.5.3. Rotating the gatekeeper webhook certificate
Complete the following steps to rotate the gatekeeper webhook certificate:
Edit the secret that contains the certificate with the following command:
oc edit secret -n openshift-gatekeeper-system gatekeeper-webhook-server-cert
-
Delete the following content in the
data
section:ca.crt
,ca.key
,tls.crt
, andtls.key
. Restart the gatekeeper webhook service by deleting the
gatekeeper-controller-manager
pods with the following command:oc delete pod -n openshift-gatekeeper-system -l control-plane=controller-manager
The gatekeeper webhook certificate is rotated.
1.2.5.4. Verifying certificate rotation
Verify that your certificates are rotated using the following steps:
- Identify the secret that you want to check.
-
Check the
tls.crt
key to verify that a certificate is available. Display the certificate information by using the following command:
oc get secret <your-secret-name> -n open-cluster-management -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout
Replace
<your-secret-name>
with the name of secret that you are verifying. If it is necessary, also update the namespace and JSON path.Check the
Validity
details in the output. View the followingValidity
example:Validity Not Before: Jul 13 15:17:50 2023 GMT 1 Not After : Jul 12 15:17:50 2024 GMT 2
1.2.5.5. Listing hub cluster managed certificates
You can view a list of hub cluster managed certificates that use OpenShift Service Serving Certificates service internally. Run the following command to list the certificates:
for ns in multicluster-engine open-cluster-management ; do echo "$ns:" ; oc get secret -n $ns -o custom-columns=Name:.metadata.name,Expiration:.metadata.annotations.service\\.beta\\.openshift\\.io/expiry | grep -v '<none>' ; echo ""; done
For more information, see OpenShift Service Serving Certificates in the Additional resources section.
Note: If observability is enabled, there are additional namespaces where certificates are created.