Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.
Chapter 3. Configuring certificates
3.1. Replacing the default ingress certificate Link kopierenLink in die Zwischenablage kopiert!
3.1.1. Understanding the default ingress certificate Link kopierenLink in die Zwischenablage kopiert!
By default, OpenShift Container Platform uses the Ingress Operator to create an internal CA and issue a wildcard certificate that is valid for applications under the
.apps
The internal infrastructure CA certificates are self-signed. While this process might be perceived as bad practice by some security or PKI teams, any risk here is minimal. The only clients that implicitly trust these certificates are other components within the cluster. Replacing the default wildcard certificate with one that is issued by a public CA already included in the CA bundle as provided by the container userspace allows external clients to connect securely to applications running under the
.apps
3.1.2. Replacing the default ingress certificate Link kopierenLink in die Zwischenablage kopiert!
You can replace the default ingress certificate for all applications under the
.apps
Prerequisites
-
You must have a wildcard certificate for the fully qualified subdomain and its corresponding private key. Each should be in a separate PEM format file.
.apps - The private key must be unencrypted. If your key is encrypted, decrypt it before importing it into OpenShift Container Platform.
-
The certificate must include the extension showing
subjectAltName.*.apps.<clustername>.<domain> - The certificate file can contain one or more certificates in a chain. The wildcard certificate must be the first certificate in the file. It can then be followed with any intermediate certificates, and the file should end with the root CA certificate.
- Copy the root CA certificate into an additional PEM format file.
-
Verify that all certificates which include also end with one carriage return after that line.
-----END CERTIFICATE-----
Updating the certificate authority (CA) causes the nodes in your cluster to reboot.
Procedure
Create a config map that includes only the root CA certificate used to sign the wildcard certificate:
$ oc create configmap custom-ca \ --from-file=ca-bundle.crt=</path/to/example-ca.crt> \1 -n openshift-config- 1
</path/to/example-ca.crt>is the path to the root CA certificate file on your local file system.
Update the cluster-wide proxy configuration with the newly created config map:
$ oc patch proxy/cluster \ --type=merge \ --patch='{"spec":{"trustedCA":{"name":"custom-ca"}}}'Create a secret that contains the wildcard certificate chain and key:
$ oc create secret tls <secret> \1 --cert=</path/to/cert.crt> \2 --key=</path/to/cert.key> \3 -n openshift-ingressUpdate the Ingress Controller configuration with the newly created secret:
$ oc patch ingresscontroller.operator default \ --type=merge -p \ '{"spec":{"defaultCertificate": {"name": "<secret>"}}}' \1 -n openshift-ingress-operator- 1
- Replace
<secret>with the name used for the secret in the previous step.
ImportantTo trigger the Ingress Operator to perform a rolling update, you must update the name of the secret. Because the kubelet automatically propagates changes to the secret in the volume mount, updating the secret contents does not trigger a rolling update. For more information, see this Red Hat Knowledgebase Solution.
Additional resources
3.2. Adding API server certificates Link kopierenLink in die Zwischenablage kopiert!
The default API server certificate is issued by an internal OpenShift Container Platform cluster CA. Clients outside of the cluster will not be able to verify the API server’s certificate by default. This certificate can be replaced by one that is issued by a CA that clients trust.
3.2.1. Add an API server named certificate Link kopierenLink in die Zwischenablage kopiert!
The default API server certificate is issued by an internal OpenShift Container Platform cluster CA. You can add one or more alternative certificates that the API server will return based on the fully qualified domain name (FQDN) requested by the client, for example when a reverse proxy or load balancer is used.
Prerequisites
- You must have a certificate for the FQDN and its corresponding private key. Each should be in a separate PEM format file.
- The private key must be unencrypted. If your key is encrypted, decrypt it before importing it into OpenShift Container Platform.
-
The certificate must include the extension showing the FQDN.
subjectAltName - The certificate file can contain one or more certificates in a chain. The certificate for the API server FQDN must be the first certificate in the file. It can then be followed with any intermediate certificates, and the file should end with the root CA certificate.
Do not provide a named certificate for the internal load balancer (host name
api-int.<cluster_name>.<base_domain>
Procedure
Login to the new API as the
user.kubeadmin$ oc login -u kubeadmin -p <password> https://FQDN:6443Get the
file.kubeconfig$ oc config view --flatten > kubeconfig-newapiCreate a secret that contains the certificate chain and private key in the
namespace.openshift-config$ oc create secret tls <secret> \1 --cert=</path/to/cert.crt> \2 --key=</path/to/cert.key> \3 -n openshift-configUpdate the API server to reference the created secret.
$ oc patch apiserver cluster \ --type=merge -p \ '{"spec":{"servingCerts": {"namedCertificates": [{"names": ["<FQDN>"], "servingCertificate": {"name": "<secret>"}}]}}}'where:
- <FQDN>
-
Replace
<FQDN>with the FQDN that the API server should provide the certificate for. Do not include the port number. - <secret>
-
Replace
<secret>with the name used for the secret in the previous step.
Examine the
object and confirm the secret is now referenced.apiserver/cluster$ oc get apiserver cluster -o yamlExample output
... spec: servingCerts: namedCertificates: - names: - <FQDN> servingCertificate: name: <secret> ...Check the
operator, and verify that a new revision of the Kubernetes API server rolls out. It may take a minute for the operator to detect the configuration change and trigger a new deployment. While the new revision is rolling out,kube-apiserverwill reportPROGRESSING.True$ oc get clusteroperators kube-apiserverDo not continue to the next step until
is listed asPROGRESSING, as shown in the following output:FalseExample output
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE kube-apiserver 4.12.0 True False False 145mIf
is showingPROGRESSING, wait a few minutes and try again.TrueNoteA new revision of the Kubernetes API server only rolls out if the API server named certificate is added for the first time. When the API server named certificate is renewed, a new revision of the Kubernetes API server does not roll out because the
pods dynamically reload the updated certificate.kube-apiserver
3.3. Securing service traffic using service serving certificate secrets Link kopierenLink in die Zwischenablage kopiert!
3.3.1. Understanding service serving certificates Link kopierenLink in die Zwischenablage kopiert!
Service serving certificates are intended to support complex middleware applications that require encryption. These certificates are issued as TLS web server certificates.
The
service-ca
x509.SHA256WithRSA
The generated certificate and key are in PEM format, stored in
tls.crt
tls.key
The service CA certificate, which issues the service certificates, is valid for 26 months and is automatically rotated when there is less than 13 months validity left. After rotation, the previous service CA configuration is still trusted until its expiration. This allows a grace period for all affected services to refresh their key material before the expiration. If you do not upgrade your cluster during this grace period, which restarts services and refreshes their key material, you might need to manually restart services to avoid failures after the previous service CA expires.
You can use the following command to manually restart all pods in the cluster. Be aware that running this command causes a service interruption, because it deletes every running pod in every namespace. These pods will automatically restart after they are deleted.
$ for I in $(oc get ns -o jsonpath='{range .items[*]} {.metadata.name}{"\n"} {end}'); \
do oc delete pods --all -n $I; \
sleep 1; \
done
3.3.2. Add a service certificate Link kopierenLink in die Zwischenablage kopiert!
To secure communication to your service, generate a signed serving certificate and key pair into a secret in the same namespace as the service.
The generated certificate is only valid for the internal service DNS name
<service.name>.<service.namespace>.svc
clusterIP
*.<service.name>.<service.namespace>.svc
Because the generated certificates contain wildcard subjects for headless services, you must not use the service CA if your client must differentiate between individual pods. In this case:
- Generate individual TLS certificates by using a different CA.
- Do not accept the service CA as a trusted CA for connections that are directed to individual pods and must not be impersonated by other pods. These connections must be configured to trust the CA that was used to generate the individual TLS certificates.
Prerequisites
- You must have a service defined.
Procedure
Annotate the service with
:service.beta.openshift.io/serving-cert-secret-name$ oc annotate service <service_name> \1 service.beta.openshift.io/serving-cert-secret-name=<secret_name>2 For example, use the following command to annotate the service
:test1$ oc annotate service test1 service.beta.openshift.io/serving-cert-secret-name=test1Examine the service to confirm that the annotations are present:
$ oc describe service <service_name>Example output
... Annotations: service.beta.openshift.io/serving-cert-secret-name: <service_name> service.beta.openshift.io/serving-cert-signed-by: openshift-service-serving-signer@1556850837 ...-
After the cluster generates a secret for your service, your spec can mount it, and the pod will run after it becomes available.
Pod
3.3.3. Add the service CA bundle to a config map Link kopierenLink in die Zwischenablage kopiert!
A pod can access the service Certificate Authority (CA) certificate by mounting a
ConfigMap
service.beta.openshift.io/inject-cabundle=true
service-ca.crt
After adding this annotation to a config map, the OpenShift Service CA Operator deletes all the data in the config map. Consider using a separate config map to contain the
service-ca.crt
Procedure
Annotate the config map with the
annotation by entering the following command:service.beta.openshift.io/inject-cabundle=true$ oc annotate configmap <config_map_name> \1 service.beta.openshift.io/inject-cabundle=true- 1
- Replace
<config_map_name>with the name of the config map to annotate.
NoteExplicitly referencing the
key in a volume mount prevents a pod from starting until the config map has been injected with the CA bundle. You can override this behavior by setting theservice-ca.crtparameter tooptionalin the serving certificate configuration of the volume.trueView the config map to ensure that the service CA bundle has been injected:
$ oc get configmap <config_map_name> -o yamlThe CA bundle is displayed as the value of the
key in the YAML output:service-ca.crtapiVersion: v1 data: service-ca.crt: | -----BEGIN CERTIFICATE----- ...Mount the config map as a volume to each container that exists in a pod by configuring your
object.DeploymentExample Deployment object that defines the volume for the mounted config map
apiVersion: apps/v1 kind: Deployment metadata: name: my-example-custom-ca-deployment namespace: my-example-custom-ca-ns spec: ... spec: ... containers: - name: my-container-that-needs-custom-ca volumeMounts: - name: trusted-ca mountPath: /etc/pki/ca-trust/extracted/pem readOnly: true volumes: - name: trusted-ca configMap: name: <config_map_name>1 items: - key: ca-bundle.crt2 path: tls-ca-bundle.pem3 # ...
3.3.4. Add the service CA bundle to an API service Link kopierenLink in die Zwischenablage kopiert!
You can annotate an
APIService
service.beta.openshift.io/inject-cabundle=true
spec.caBundle
Procedure
Annotate the API service with
:service.beta.openshift.io/inject-cabundle=true$ oc annotate apiservice <api_service_name> \1 service.beta.openshift.io/inject-cabundle=true- 1
- Replace
<api_service_name>with the name of the API service to annotate.
For example, use the following command to annotate the API service
:test1$ oc annotate apiservice test1 service.beta.openshift.io/inject-cabundle=trueView the API service to ensure that the service CA bundle has been injected:
$ oc get apiservice <api_service_name> -o yamlThe CA bundle is displayed in the
field in the YAML output:spec.caBundleapiVersion: apiregistration.k8s.io/v1 kind: APIService metadata: annotations: service.beta.openshift.io/inject-cabundle: "true" ... spec: caBundle: <CA_BUNDLE> ...
3.3.5. Add the service CA bundle to a custom resource definition Link kopierenLink in die Zwischenablage kopiert!
You can annotate a
CustomResourceDefinition
service.beta.openshift.io/inject-cabundle=true
spec.conversion.webhook.clientConfig.caBundle
The service CA bundle will only be injected into the CRD if the CRD is configured to use a webhook for conversion. It is only useful to inject the service CA bundle if a CRD’s webhook is secured with a service CA certificate.
Procedure
Annotate the CRD with
:service.beta.openshift.io/inject-cabundle=true$ oc annotate crd <crd_name> \1 service.beta.openshift.io/inject-cabundle=true- 1
- Replace
<crd_name>with the name of the CRD to annotate.
For example, use the following command to annotate the CRD
:test1$ oc annotate crd test1 service.beta.openshift.io/inject-cabundle=trueView the CRD to ensure that the service CA bundle has been injected:
$ oc get crd <crd_name> -o yamlThe CA bundle is displayed in the
field in the YAML output:spec.conversion.webhook.clientConfig.caBundleapiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: service.beta.openshift.io/inject-cabundle: "true" ... spec: conversion: strategy: Webhook webhook: clientConfig: caBundle: <CA_BUNDLE> ...
3.3.6. Add the service CA bundle to a mutating webhook configuration Link kopierenLink in die Zwischenablage kopiert!
You can annotate a
MutatingWebhookConfiguration
service.beta.openshift.io/inject-cabundle=true
clientConfig.caBundle
Do not set this annotation for admission webhook configurations that need to specify different CA bundles for different webhooks. If you do, then the service CA bundle will be injected for all webhooks.
Procedure
Annotate the mutating webhook configuration with
:service.beta.openshift.io/inject-cabundle=true$ oc annotate mutatingwebhookconfigurations <mutating_webhook_name> \1 service.beta.openshift.io/inject-cabundle=true- 1
- Replace
<mutating_webhook_name>with the name of the mutating webhook configuration to annotate.
For example, use the following command to annotate the mutating webhook configuration
:test1$ oc annotate mutatingwebhookconfigurations test1 service.beta.openshift.io/inject-cabundle=trueView the mutating webhook configuration to ensure that the service CA bundle has been injected:
$ oc get mutatingwebhookconfigurations <mutating_webhook_name> -o yamlThe CA bundle is displayed in the
field of all webhooks in the YAML output:clientConfig.caBundleapiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration metadata: annotations: service.beta.openshift.io/inject-cabundle: "true" ... webhooks: - myWebhook: - v1beta1 clientConfig: caBundle: <CA_BUNDLE> ...
3.3.7. Add the service CA bundle to a validating webhook configuration Link kopierenLink in die Zwischenablage kopiert!
You can annotate a
ValidatingWebhookConfiguration
service.beta.openshift.io/inject-cabundle=true
clientConfig.caBundle
Do not set this annotation for admission webhook configurations that need to specify different CA bundles for different webhooks. If you do, then the service CA bundle will be injected for all webhooks.
Procedure
Annotate the validating webhook configuration with
:service.beta.openshift.io/inject-cabundle=true$ oc annotate validatingwebhookconfigurations <validating_webhook_name> \1 service.beta.openshift.io/inject-cabundle=true- 1
- Replace
<validating_webhook_name>with the name of the validating webhook configuration to annotate.
For example, use the following command to annotate the validating webhook configuration
:test1$ oc annotate validatingwebhookconfigurations test1 service.beta.openshift.io/inject-cabundle=trueView the validating webhook configuration to ensure that the service CA bundle has been injected:
$ oc get validatingwebhookconfigurations <validating_webhook_name> -o yamlThe CA bundle is displayed in the
field of all webhooks in the YAML output:clientConfig.caBundleapiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: annotations: service.beta.openshift.io/inject-cabundle: "true" ... webhooks: - myWebhook: - v1beta1 clientConfig: caBundle: <CA_BUNDLE> ...
3.3.8. Manually rotate the generated service certificate Link kopierenLink in die Zwischenablage kopiert!
You can rotate the service certificate by deleting the associated secret. Deleting the secret results in a new one being automatically created, resulting in a new certificate.
Prerequisites
- A secret containing the certificate and key pair must have been generated for the service.
Procedure
Examine the service to determine the secret containing the certificate. This is found in the
annotation, as seen below.serving-cert-secret-name$ oc describe service <service_name>Example output
... service.beta.openshift.io/serving-cert-secret-name: <secret> ...Delete the generated secret for the service. This process will automatically recreate the secret.
$ oc delete secret <secret>1 - 1
- Replace
<secret>with the name of the secret from the previous step.
Confirm that the certificate has been recreated by obtaining the new secret and examining the
.AGE$ oc get secret <service_name>Example output
NAME TYPE DATA AGE <service.name> kubernetes.io/tls 2 1s
3.3.9. Manually rotate the service CA certificate Link kopierenLink in die Zwischenablage kopiert!
The service CA is valid for 26 months and is automatically refreshed when there is less than 13 months validity left.
If necessary, you can manually refresh the service CA by using the following procedure.
A manually-rotated service CA does not maintain trust with the previous service CA. You might experience a temporary service disruption until the pods in the cluster are restarted, which ensures that pods are using service serving certificates issued by the new service CA.
Prerequisites
- You must be logged in as a cluster admin.
Procedure
View the expiration date of the current service CA certificate by using the following command.
$ oc get secrets/signing-key -n openshift-service-ca \ -o template='{{index .data "tls.crt"}}' \ | base64 --decode \ | openssl x509 -noout -enddateManually rotate the service CA. This process generates a new service CA which will be used to sign the new service certificates.
$ oc delete secret/signing-key -n openshift-service-caTo apply the new certificates to all services, restart all the pods in your cluster. This command ensures that all services use the updated certificates.
$ for I in $(oc get ns -o jsonpath='{range .items[*]} {.metadata.name}{"\n"} {end}'); \ do oc delete pods --all -n $I; \ sleep 1; \ doneWarningThis command will cause a service interruption, as it goes through and deletes every running pod in every namespace. These pods will automatically restart after they are deleted.
3.4. Updating the CA bundle Link kopierenLink in die Zwischenablage kopiert!
Updating the certificate authority (CA) will cause the nodes of your cluster to reboot.
3.4.1. Understanding the CA Bundle certificate Link kopierenLink in die Zwischenablage kopiert!
Proxy certificates allow users to specify one or more custom certificate authority (CA) used by platform components when making egress connections.
The
trustedCA
image-registry-operator
trustedCA
The
trustedCA
ca-bundle.crt
trusted-ca-bundle
openshift-config-managed
trustedCA
openshift-config
apiVersion: v1
kind: ConfigMap
metadata:
name: user-ca-bundle
namespace: openshift-config
data:
ca-bundle.crt: |
-----BEGIN CERTIFICATE-----
Custom CA certificate bundle.
-----END CERTIFICATE-----
3.4.2. Replacing the CA Bundle certificate Link kopierenLink in die Zwischenablage kopiert!
Procedure
Create a config map that includes the root CA certificate used to sign the wildcard certificate:
$ oc create configmap custom-ca \ --from-file=ca-bundle.crt=</path/to/example-ca.crt> \1 -n openshift-config- 1
</path/to/example-ca.crt>is the path to the CA certificate bundle on your local file system.
Update the cluster-wide proxy configuration with the newly created config map:
$ oc patch proxy/cluster \ --type=merge \ --patch='{"spec":{"trustedCA":{"name":"custom-ca"}}}'