Chapter 9. cert-manager Operator for Red Hat OpenShift

Procedure

1. Log in to the OpenShift Container Platform web console.
2. Navigate to Operators OperatorHub.
3. Enter cert-manager Operator for Red Hat OpenShift into the filter box.

4. Select the cert-manager Operator for Red Hat OpenShift and click Install.

   Note

   From the cert-manager Operator for Red Hat OpenShift

   1.12.0

   and later, the z-stream versions of the upstream cert-manager operands such as cert-manager controller, CA injector, Webhook, and cert-manager Operator for Red Hat OpenShift are decoupled. For example, for the cert-manager Operator for Red Hat OpenShift

   1.12.0

   , the cert-manager operand version is

   v1.12.4

   .

5. On the Install Operator page:

   1. Update the Update channel, if necessary. The channel defaults to stable-v1, which installs the latest stable release of the cert-manager Operator for Red Hat OpenShift.

   2. Choose the Installed Namespace for the Operator. The default Operator namespace is

      cert-manager-operator

      .

      If the

      cert-manager-operator

      namespace does not exist, it is created for you.
      Note

      During the installation, the OpenShift Container Platform web console allows you to select between

      AllNamespaces

      and

      SingleNamespace

      installation modes. For installations with cert-manager Operator for Red Hat OpenShift version 1.15.0 or later, it is recommended to choose the

      AllNamespaces

      installation mode.

      SingleNamespace

      and

      OwnNamespace

      support will remain for earlier versions but will be deprecated in future versions.

   3. Select an Update approval strategy.

      • The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
      • The Manual strategy requires a user with appropriate credentials to approve the Operator update.
   4. Click Install.

Verification

1. Navigate to Operators Installed Operators.
2. Verify that cert-manager Operator for Red Hat OpenShift is listed with a Status of Succeeded in the

   cert-manager-operator

   namespace.

3. Verify that cert-manager pods are up and running by entering the following command:

   $ oc get pods -n cert-manager

   Example output

   NAME                                       READY   STATUS    RESTARTS   AGE
   cert-manager-bd7fbb9fc-wvbbt               1/1     Running   0          3m39s
   cert-manager-cainjector-56cc5f9868-7g9z7   1/1     Running   0          4m5s
   cert-manager-webhook-d4f79d7f7-9dg9w       1/1     Running   0          4m9s

   You can use the cert-manager Operator for Red Hat OpenShift only after cert-manager pods are up and running.

Procedure

1. Create an ACME cluster issuer.

   1. Create a YAML file that defines the

      ClusterIssuer

      object:

      Example acme-cluster-issuer.yaml file

      apiVersion: cert-manager.io/v1
      kind: ClusterIssuer
      metadata:
        name: letsencrypt-staging                                        

      1

      
      spec:
        acme:
          preferredChain: ""
          privateKeySecretRef:
            name: <secret_for_private_key>                               

      2

      
          server: https://acme-staging-v02.api.letsencrypt.org/directory 

      3

      
          solvers:
          - http01:
              ingress:
                ingressClassName: openshift-default                                 

      4

      1

      Provide a name for the cluster issuer.

      2

      Replace <secret_private_key> with the name of secret to store the ACME account private key in.

      3

      Specify the URL to access the ACME server’s directory endpoint. This example uses the Let’s Encrypt staging environment.

      4

      Specify the Ingress class.

   2. Optional: If you create the object without specifying

      ingressClassName

      , use the following command to patch the existing ingress:

      $ oc patch ingress/<ingress-name> --type=merge --patch '{"spec":{"ingressClassName":"openshift-default"}}' -n <namespace>

   3. Create the

      ClusterIssuer

      object by running the following command:

      $ oc create -f acme-cluster-issuer.yaml

2. Create an Ingress to expose the service of the user workload.

   1. Create a YAML file that defines a

      Namespace

      object:

      Example namespace.yaml file

      apiVersion: v1
      kind: Namespace
      metadata:
        name: my-ingress-namespace 

      1

      1

      Specify the namespace for the Ingress.

   2. Create the

      Namespace

      object by running the following command:

      $ oc create -f namespace.yaml

   3. Create a YAML file that defines the

      Ingress

      object:

      Example ingress.yaml file

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: sample-ingress                                           

      1

      
        namespace: my-ingress-namespace                                

      2

      
        annotations:
          cert-manager.io/cluster-issuer: letsencrypt-staging          

      3

      
      spec:
        ingressClassName: openshift-default                            

      4

      
        tls:
        - hosts:
          - <hostname>                                                 

      5

      
          secretName: sample-tls                                       

      6

      
        rules:
        - host: <hostname>                                             

      7

      
          http:
            paths:
            - path: /
              pathType: Prefix
              backend:
                service:
                  name: sample-workload                                

      8

      
                  port:
                    number: 80

      1

      Specify the name of the Ingress.

      2

      Specify the namespace that you created for the Ingress.

      3

      Specify the cluster issuer that you created.

      4

      Specify the Ingress class.

      5

      Replace <hostname> with the Subject Alternative Name (SAN) to be associated with the certificate. This name is used to add DNS names to the certificate.

      6

      Specify the secret that stores the certificate.

      7

      Replace <hostname> with the hostname. You can use the <host_name>.<cluster_ingress_domain> syntax to take advantage of the *.<cluster_ingress_domain> wildcard DNS record and serving certificate for the cluster. For example, you might use apps.<cluster_base_domain>. Otherwise, you must ensure that a DNS record exists for the chosen hostname.

      8

      Specify the name of the service to expose. This example uses a service named sample-workload.

   4. Create the

      Ingress

      object by running the following command:

      $ oc create -f ingress.yaml

Procedure

1. Optional: Override the nameserver settings for the DNS-01 self check.

   This step is required only when the target public-hosted zone overlaps with the cluster’s default private-hosted zone.

   1. Edit the

      CertManager

      resource by running the following command:

      $ oc edit certmanager cluster

   2. Add a

      spec.controllerConfig

      section with the following override arguments:

      apiVersion: operator.openshift.io/v1alpha1
      kind: CertManager
      metadata:
        name: cluster
        ...
      spec:
        ...
        controllerConfig:                                

      1

      
          overrideArgs:
            - '--dns01-recursive-nameservers-only'       

      2

      
            - '--dns01-recursive-nameservers=1.1.1.1:53' 

      3

      1

      Add the spec.controllerConfig section.

      2

      Specify to only use recursive nameservers instead of checking the authoritative nameservers associated with that domain.

      3

      Provide a comma-separated list of <host>:<port> nameservers to query for the DNS-01 self check. You must use a 1.1.1.1:53 value to avoid the public and private zones overlapping.

   3. Save the file to apply the changes.

2. Optional: Create a namespace for the issuer:

   $ oc new-project <issuer_namespace>

3. Create a secret to store your AWS credentials in by running the following command:

   $ oc create secret generic aws-secret --from-literal=awsSecretAccessKey=<aws_secret_access_key> \ 

   1

   
       -n my-issuer-namespace

   1

   Replace <aws_secret_access_key> with your AWS secret access key.

4. Create an issuer:

   1. Create a YAML file that defines the

      Issuer

      object:

      Example issuer.yaml file

      apiVersion: cert-manager.io/v1
      kind: Issuer
      metadata:
        name: <letsencrypt_staging>                                        

      1

      
        namespace: <issuer_namespace>                                   

      2

      
      spec:
        acme:
          server: https://acme-staging-v02.api.letsencrypt.org/directory 

      3

      
          email: "<email_address>"                                       

      4

      
          privateKeySecretRef:
            name: <secret_private_key>                                   

      5

      
          solvers:
          - dns01:
              route53:
                accessKeyID: <aws_key_id>                                

      6

      
                hostedZoneID: <hosted_zone_id>                           

      7

      
                region: <region_name>                                    

      8

      
                secretAccessKeySecretRef:
                  name: "aws-secret"                                     

      9

      
                  key: "awsSecretAccessKey"                              

      10

      1

      Provide a name for the issuer.

      2

      Specify the namespace that you created for the issuer.

      3

      Specify the URL to access the ACME server’s directory endpoint. This example uses the Let’s Encrypt staging environment.

      4

      Replace <email_address> with your email address.

      5

      Replace <secret_private_key> with the name of the secret to store the ACME account private key in.

      6

      Replace <aws_key_id> with your AWS key ID.

      7

      Replace <hosted_zone_id> with your hosted zone ID.

      8

      Replace <region_name> with the AWS region name. For example, us-east-1.

      9

      Specify the name of the secret you created.

      10

      Specify the key in the secret you created that stores your AWS secret access key.

   2. Create the

      Issuer

      object by running the following command:

      $ oc create -f issuer.yaml

Procedure

1. Optional: Override the nameserver settings for the DNS-01 self check.

   This step is required only when the target public-hosted zone overlaps with the cluster’s default private-hosted zone.

   1. Edit the

      CertManager

      resource by running the following command:

      $ oc edit certmanager cluster

   2. Add a

      spec.controllerConfig

      section with the following override arguments:

      apiVersion: operator.openshift.io/v1alpha1
      kind: CertManager
      metadata:
        name: cluster
        ...
      spec:
        ...
        controllerConfig:                                

      1

      
          overrideArgs:
            - '--dns01-recursive-nameservers-only'       

      2

      
            - '--dns01-recursive-nameservers=1.1.1.1:53' 

      3

      1

      Add the spec.controllerConfig section.

      2

      Specify to only use recursive nameservers instead of checking the authoritative nameservers associated with that domain.

      3

      Provide a comma-separated list of <host>:<port> nameservers to query for the DNS-01 self check. You must use a 1.1.1.1:53 value to avoid the public and private zones overlapping.

   3. Save the file to apply the changes.

2. Optional: Create a namespace for the issuer:

   $ oc new-project <issuer_namespace>

3. Modify the

   CertManager

   resource to add the

   --issuer-ambient-credentials

   argument:

   $ oc patch certmanager/cluster \
     --type=merge \
     -p='{"spec":{"controllerConfig":{"overrideArgs":["--issuer-ambient-credentials"]}}}'

4. Create an issuer:

   1. Create a YAML file that defines the

      Issuer

      object:

      Example issuer.yaml file

      apiVersion: cert-manager.io/v1
      kind: Issuer
      metadata:
        name: <letsencrypt_staging>                                        

      1

      
        namespace: <issuer_namespace>                                   

      2

      
      spec:
        acme:
          server: https://acme-staging-v02.api.letsencrypt.org/directory 

      3

      
          email: "<email_address>"                                       

      4

      
          privateKeySecretRef:
            name: <secret_private_key>                                   

      5

      
          solvers:
          - dns01:
              route53:
                hostedZoneID: <hosted_zone_id>                           

      6

      
                region: us-east-1

      1

      Provide a name for the issuer.

      2

      Specify the namespace that you created for the issuer.

      3

      Specify the URL to access the ACME server’s directory endpoint. This example uses the Let’s Encrypt staging environment.

      4

      Replace <email_address> with your email address.

      5

      Replace <secret_private_key> with the name of the secret to store the ACME account private key in.

      6

      Replace <hosted_zone_id> with your hosted zone ID.

   2. Create the

      Issuer

      object by running the following command:

      $ oc create -f issuer.yaml

Procedure

1. Optional: Override the nameserver settings for the DNS-01 self check.

   This step is required only when the target public-hosted zone overlaps with the cluster’s default private-hosted zone.

   1. Edit the

      CertManager

      resource by running the following command:

      $ oc edit certmanager cluster

   2. Add a

      spec.controllerConfig

      section with the following override arguments:

      apiVersion: operator.openshift.io/v1alpha1
      kind: CertManager
      metadata:
        name: cluster
        ...
      spec:
        ...
        controllerConfig:                                

      1

      
          overrideArgs:
            - '--dns01-recursive-nameservers-only'       

      2

      
            - '--dns01-recursive-nameservers=1.1.1.1:53' 

      3

      1

      Add the spec.controllerConfig section.

      2

      Specify to only use recursive nameservers instead of checking the authoritative nameservers associated with that domain.

      3

      Provide a comma-separated list of <host>:<port> nameservers to query for the DNS-01 self check. You must use a 1.1.1.1:53 value to avoid the public and private zones overlapping.

   3. Save the file to apply the changes.

2. Optional: Create a namespace for the issuer:

   $ oc new-project my-issuer-namespace

3. Create a secret to store your Google Cloud credentials by running the following command:

   $ oc create secret generic clouddns-dns01-solver-svc-acct --from-file=service_account.json=<path/to/gcp_service_account.json> -n my-issuer-namespace

4. Create an issuer:

   1. Create a YAML file that defines the

      Issuer

      object:

      Example issuer.yaml file

      apiVersion: cert-manager.io/v1
      kind: Issuer
      metadata:
        name: <acme_dns01_clouddns_issuer> 

      1

      
        namespace: <issuer_namespace> 

      2

      
      spec:
        acme:
          preferredChain: ""
          privateKeySecretRef:
            name: <secret_private_key> 

      3

      
          server: https://acme-staging-v02.api.letsencrypt.org/directory 

      4

      
          solvers:
          - dns01:
              cloudDNS:
                project: <project_id> 

      5

      
                serviceAccountSecretRef:
                  name: clouddns-dns01-solver-svc-acct 

      6

      
                  key: service_account.json 

      7

      1

      Provide a name for the issuer.

      2

      Replace <issuer_namespace> with your issuer namespace.

      3

      Replace <secret_private_key> with the name of the secret to store the ACME account private key in.

      4

      Specify the URL to access the ACME server’s directory endpoint. This example uses the Let’s Encrypt staging environment.

      5

      Replace <project_id> with the name of the Google Cloud project that contains the Cloud DNS zone.

      6

      Specify the name of the secret you created.

      7

      Specify the key in the secret you created that stores your Google Cloud secret access key.

   2. Create the

      Issuer

      object by running the following command:

      $ oc create -f issuer.yaml

Procedure

1. Optional: Override the nameserver settings for the DNS-01 self check.

   This step is required only when the target public-hosted zone overlaps with the cluster’s default private-hosted zone.

   1. Edit the

      CertManager

      resource by running the following command:

      $ oc edit certmanager cluster

   2. Add a

      spec.controllerConfig

      section with the following override arguments:

      apiVersion: operator.openshift.io/v1alpha1
      kind: CertManager
      metadata:
        name: cluster
        ...
      spec:
        ...
        controllerConfig:                                

      1

      
          overrideArgs:
            - '--dns01-recursive-nameservers-only'       

      2

      
            - '--dns01-recursive-nameservers=1.1.1.1:53' 

      3

      1

      Add the spec.controllerConfig section.

      2

      Specify to only use recursive nameservers instead of checking the authoritative nameservers associated with that domain.

      3

      Provide a comma-separated list of <host>:<port> nameservers to query for the DNS-01 self check. You must use a 1.1.1.1:53 value to avoid the public and private zones overlapping.

   3. Save the file to apply the changes.

2. Optional: Create a namespace for the issuer:

   $ oc new-project <issuer_namespace>

3. Modify the

   CertManager

   resource to add the

   --issuer-ambient-credentials

   argument:

   $ oc patch certmanager/cluster \
     --type=merge \
     -p='{"spec":{"controllerConfig":{"overrideArgs":["--issuer-ambient-credentials"]}}}'

4. Create an issuer:

   1. Create a YAML file that defines the

      Issuer

      object:

      Example issuer.yaml file

      apiVersion: cert-manager.io/v1
      kind: Issuer
      metadata:
        name: <acme_dns01_clouddns_issuer> 

      1

      
        namespace: <issuer_namespace>
      spec:
        acme:
          preferredChain: ""
          privateKeySecretRef:
            name: <secret_private_key> 

      2

      
          server: https://acme-staging-v02.api.letsencrypt.org/directory 

      3

      
          solvers:
          - dns01:
              cloudDNS:
                project: <gcp_project_id> 

      4

      1

      Provide a name for the issuer.

      2

      Replace <secret_private_key> with the name of the secret to store the ACME account private key in.

      3

      Specify the URL to access the ACME server’s directory endpoint. This example uses the Let’s Encrypt staging environment.

      4

      Replace <gcp_project_id> with the name of the Google Cloud project that contains the Cloud DNS zone.

   2. Create the

      Issuer

      object by running the following command:

      $ oc create -f issuer.yaml

Procedure

1. Optional: Override the nameserver settings for the DNS-01 self check.

   This step is required only when the target public-hosted zone overlaps with the cluster’s default private-hosted zone.

   1. Edit the

      CertManager

      resource by running the following command:

      $ oc edit certmanager cluster

   2. Add a

      spec.controllerConfig

      section with the following override arguments:

      apiVersion: operator.openshift.io/v1alpha1
      kind: CertManager
      metadata:
        name: cluster
        ...
      spec:
        ...
        controllerConfig:                                

      1

      
          overrideArgs:
            - '--dns01-recursive-nameservers-only'       

      2

      
            - '--dns01-recursive-nameservers=1.1.1.1:53' 

      3

      1

      Add the spec.controllerConfig section.

      2

      Specify to only use recursive nameservers instead of checking the authoritative nameservers associated with that domain.

      3

      Provide a comma-separated list of <host>:<port> nameservers to query for the DNS-01 self check. You must use a 1.1.1.1:53 value to avoid the public and private zones overlapping.

   3. Save the file to apply the changes.

2. Optional: Create a namespace for the issuer:

   $ oc new-project my-issuer-namespace

3. Create a secret to store your Azure credentials in by running the following command:

   $ oc create secret generic <secret_name> --from-literal=<azure_secret_access_key_name>=<azure_secret_access_key_value> \ 

   1

   2

   3

   
       -n my-issuer-namespace

   1

   Replace <secret_name> with your secret name.

   2

   Replace <azure_secret_access_key_name> with your Azure secret access key name.

   3

   Replace <azure_secret_access_key_value> with your Azure secret key.

4. Create an issuer:

   1. Create a YAML file that defines the

      Issuer

      object:

      Example issuer.yaml file

      apiVersion: cert-manager.io/v1
      kind: Issuer
      metadata:
        name: <acme-dns01-azuredns-issuer>   

      1

      
        namespace: <issuer_namespace>   

      2

      
      spec:
        acme:
          preferredChain: ""
          privateKeySecretRef:
            name: <secret_private_key> 

      3

      
          server: https://acme-staging-v02.api.letsencrypt.org/directory 

      4

      
          solvers:
          - dns01:
              azureDNS:
                clientID: <azure_client_id> 

      5

      
                clientSecretSecretRef:
                  name: <secret_name> 

      6

      
                  key: <azure_secret_access_key_name> 

      7

      
                subscriptionID: <azure_subscription_id> 

      8

      
                tenantID: <azure_tenant_id> 

      9

      
                resourceGroupName: <azure_dns_zone_resource_group> 

      10

      
                hostedZoneName: <azure_dns_zone> 

      11

      
                environment: AzurePublicCloud

      1

      Provide a name for the issuer.

      2

      Replace <issuer_namespace> with your issuer namespace.

      3

      Replace <secret_private_key> with the name of the secret to store the ACME account private key in.

      4

      Specify the URL to access the ACME server’s directory endpoint. This example uses the Let’s Encrypt staging environment.

      5

      Replace <azure_client_id> with your Azure client ID.

      6

      Replace <secret_name> with a name of the client secret.

      7

      Replace <azure_secret_access_key_name> with the client secret key name.

      8

      Replace <azure_subscription_id> with your Azure subscription ID.

      9

      Replace <azure_tenant_id> with your Azure tenant ID.

      10

      Replace <azure_dns_zone_resource_group> with the name of the Azure DNS zone resource group.

      11

      Replace <azure_dns_zone> with the name of Azure DNS zone.

   2. Create the

      Issuer

      object by running the following command:

      $ oc create -f issuer.yaml

Procedure

1. Create an issuer. For more information, see "Configuring an issuer" in the "Additional resources" section.

2. Create a certificate:

   1. Create a YAML file, for example,

      certificate.yaml

      , that defines the

      Certificate

      object:

      Example certificate.yaml file

      apiVersion: cert-manager.io/v1
      kind: Certificate
      metadata:
        name: <tls_cert> 

      1

      
        namespace: <issuer_namespace> 

      2

      
      spec:
        isCA: false
        commonName: '<common_name>' 

      3

      
        secretName: <secret_name> 

      4

      
        dnsNames:
        - "<domain_name>" 

      5

      
        issuerRef:
          name: <issuer_name> 

      6

      
          kind: Issuer

      1

      Provide a name for the certificate.

      2

      Specify the namespace of the issuer.

      3

      Specify the common name (CN).

      4

      Specify the name of the secret to create that contains the certificate.

      5

      Specify the domain name.

      6

      Specify the name of the issuer.

   2. Create the

      Certificate

      object by running the following command:

      $ oc create -f certificate.yaml

Procedure

1. Create an issuer. For more information, see "Configuring an issuer" in the "Additional resources" section.

2. Create a certificate:

   1. Create a YAML file, for example,

      certificate.yaml

      , that defines the

      Certificate

      object:

      Example certificate.yaml file

      apiVersion: cert-manager.io/v1
      kind: Certificate
      metadata:
        name: <tls_cert> 

      1

      
        namespace: openshift-config
      spec:
        isCA: false
        commonName: "api.<cluster_base_domain>" 

      2

      
        secretName: <secret_name> 

      3

      
        dnsNames:
        - "api.<cluster_base_domain>" 

      4

      
        issuerRef:
          name: <issuer_name> 

      5

      
          kind: Issuer

      1

      Provide a name for the certificate.

      2

      Specify the common name (CN).

      3

      Specify the name of the secret to create that contains the certificate.

      4

      Specify the DNS name of the API server.

      5

      Specify the name of the issuer.

   2. Create the

      Certificate

      object by running the following command:

      $ oc create -f certificate.yaml

3. Add the API server named certificate. For more information, see "Adding an API server named certificate" section in the "Additional resources" section.

Procedure

1. Create an issuer. For more information, see "Configuring an issuer" in the "Additional resources" section.

2. Create a certificate:

   1. Create a YAML file, for example,

      certificate.yaml

      , that defines the

      Certificate

      object:

      Example certificate.yaml file

      apiVersion: cert-manager.io/v1
      kind: Certificate
      metadata:
        name: <tls_cert> 

      1

      
        namespace: openshift-ingress
      spec:
        isCA: false
        commonName: "apps.<cluster_base_domain>" 

      2

      
        secretName: <secret_name> 

      3

      
        dnsNames:
        - "apps.<cluster_base_domain>" 

      4

      
        - "*.apps.<cluster_base_domain>" 

      5

      
        issuerRef:
          name: <issuer_name> 

      6

      
          kind: Issuer

      1

      Provide a name for the certificate.

      2

      Specify the common name (CN).

      3

      Specify the name of the secret to create that contains the certificate.

      4 5

      Specify the DNS name of the ingress.

      6

      Specify the name of the issuer.

   2. Create the

      Certificate

      object by running the following command:

      $ oc create -f certificate.yaml

3. Replace the default ingress certificate. For more information, see "Replacing the default ingress certificate" section in the "Additional resources" section.

Procedure

1. Create the

   cluster-monitoring-config.yaml

   YAML file:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: cluster-monitoring-config
     namespace: openshift-monitoring
   data:
     config.yaml: |
       enableUserWorkload: true

2. Apply the

   ConfigMap

   by running the following command:

   $ oc apply -f cluster-monitoring-config.yaml

Verification

1. Verify that the monitoring components for user workloads are running in the

   openshift-user-workload-monitoring

   namespace by running the following command:

   $ oc -n openshift-user-workload-monitoring get pod

   Example output

   NAME                                   READY   STATUS    RESTARTS   AGE
   prometheus-operator-6cb6bd9588-dtzxq   2/2     Running   0          50s
   prometheus-user-workload-0             6/6     Running   0          48s
   prometheus-user-workload-1             6/6     Running   0          48s
   thanos-ruler-user-workload-0           4/4     Running   0          42s
   thanos-ruler-user-workload-1           4/4     Running   0          42s

   The status of the pods such as

   prometheus-operator

   ,

   prometheus-user-workload

   , and

   thanos-ruler-user-workload

   must be

   Running

   .

Procedure

1. Create the

   ServiceMonitor

   CR:

   1. Create the YAML file that defines the

      ServiceMonitor

      CR:

      Example servicemonitor-cert-manager.yaml file

      apiVersion: monitoring.coreos.com/v1
      kind: ServiceMonitor
      metadata:
        labels:
          app: cert-manager
          app.kubernetes.io/instance: cert-manager
          app.kubernetes.io/name: cert-manager
        name: cert-manager
        namespace: cert-manager
      spec:
        endpoints:
          - honorLabels: false
            interval: 60s
            path: /metrics
            scrapeTimeout: 30s
            targetPort: 9402
        selector:
          matchExpressions:
            - key: app.kubernetes.io/name
              operator: In
              values:
                - cainjector
                - cert-manager
                - webhook
            - key: app.kubernetes.io/instance
              operator: In
              values:
                - cert-manager
            - key: app.kubernetes.io/component
              operator: In
              values:
                - cainjector
                - controller
                - webhook

   2. Create the

      ServiceMonitor

      CR by running the following command:

      $ oc apply -f servicemonitor-cert-manager.yaml

      After the

      ServiceMonitor

      CR is created, the user workload Prometheus instance begins metrics collection from the cert-manager Operator for Red Hat OpenShift operands. The collected metrics are labeled with

      job="cert-manager"

      ,

      job="cert-manager-cainjector"

      , and

      job="cert-manager-webhook"

      .

Verification

1. In the OpenShift Container Platform web console, navigate to Observe Targets.

2. In the Label filter field, enter the following labels to filter the metrics targets for each operand:

   $ service=cert-manager

   $ service=cert-manager-webhook

   $ service=cert-manager-cainjector

3. Confirm that the Status column shows

   Up

   for the

   cert-manager

   ,

   cert-manager-webhook

   , and

   cert-manager-cainjector

   entries.

Procedure

1. In the OpenShift Container Platform web console, navigate to Observe Metrics.

2. In the query field, enter the following PromQL expressions to query the cert-manager Operator for Red Hat OpenShift operands metric for each operand:

   {job="cert-manager"}

   {job="cert-manager-webhook"}

   {job="cert-manager-cainjector"}

Procedure

1. Remove the

   IstioCSR

   custom resource by running the following command:

   $ oc -n <istio_csr_project_name> delete istiocsrs.operator.openshift.io default

2. Remove related resources:

   Important

   To avoid disrupting any Red Hat OpenShift Service Mesh or Istio components, ensure that no component is referencing the Istio-CSR service or the certificates issued for Istio before removing the following resources.

   1. List the cluster scoped-resources by running the following command and save the names of the listed resources for later reference:

      $ oc get clusterrolebindings,clusterroles -l "app=cert-manager-istio-csr,app.kubernetes.io/name=cert-manager-istio-csr"

   2. List the resources in Istio-csr deployed namespace by running the following command and save the names of the listed resources for later reference:

      $ oc get certificate,deployments,services,serviceaccounts -l "app=cert-manager-istio-csr,app.kubernetes.io/name=cert-manager-istio-csr" -n <istio_csr_project_name>

   3. List the resources in Red Hat OpenShift Service Mesh or Istio deployed namespaces by running the following command and save the names of the listed resources for later reference:

      $ oc get roles,rolebindings -l "app=cert-manager-istio-csr,app.kubernetes.io/name=cert-manager-istio-csr" -n <istio_csr_project_name>

   4. For each resource listed in previous steps, delete the resource by running the following command:

      $ oc -n <istio_csr_project_name> delete <resource_type>/<resource_name>

      Repeat this process until all of the related resources have been deleted.

Procedure

1. Create a config map in the

   cert-manager

   namespace by running the following command:

   $ oc create configmap trusted-ca -n cert-manager

2. Inject the CA bundle that is trusted by OpenShift Container Platform into the config map by running the following command:

   $ oc label cm trusted-ca config.openshift.io/inject-trusted-cabundle=true -n cert-manager

3. Update the deployment for the cert-manager Operator for Red Hat OpenShift to use the config map by running the following command:

   $ oc -n cert-manager-operator patch subscription openshift-cert-manager-operator --type='merge' -p '{"spec":{"config":{"env":[{"name":"TRUSTED_CA_CONFIGMAP_NAME","value":"trusted-ca"}]}}}'

Verification

1. Verify that the deployments have finished rolling out by running the following command:

   $ oc rollout status deployment/cert-manager-operator-controller-manager -n cert-manager-operator && \
   oc rollout status deployment/cert-manager -n cert-manager && \
   oc rollout status deployment/cert-manager-webhook -n cert-manager && \
   oc rollout status deployment/cert-manager-cainjector -n cert-manager

   Example output

   deployment "cert-manager-operator-controller-manager" successfully rolled out
   deployment "cert-manager" successfully rolled out
   deployment "cert-manager-webhook" successfully rolled out
   deployment "cert-manager-cainjector" successfully rolled out

2. Verify that the CA bundle was mounted as a volume by running the following command:

   $ oc get deployment cert-manager -n cert-manager -o=jsonpath={.spec.template.spec.'containers[0].volumeMounts'}

   Example output

   [{"mountPath":"/etc/pki/tls/certs/cert-manager-tls-ca-bundle.crt","name":"trusted-ca","subPath":"ca-bundle.crt"}]

3. Verify that the source of the CA bundle is the

   trusted-ca

   config map by running the following command:

   $ oc get deployment cert-manager -n cert-manager -o=jsonpath={.spec.template.spec.volumes}

   Example output

   [{"configMap":{"defaultMode":420,"name":"trusted-ca"},"name":"trusted-ca"}]

Procedure

1. Edit the

   CertManager

   resource by running the following command:

   $ oc edit certmanager cluster

2. Add a

   spec.controllerConfig

   section with the following override arguments:

   apiVersion: operator.openshift.io/v1alpha1
   kind: CertManager
   metadata:
     name: cluster
     ...
   spec:
     ...
     controllerConfig:
       overrideEnv:
         - name: HTTP_PROXY
           value: http://<proxy_url> 

   1

   
         - name: HTTPS_PROXY
           value: https://<proxy_url> 

   2

   
         - name: NO_PROXY
           value: <ignore_proxy_domains> 

   3

   1 2

   Replace <proxy_url> with the proxy server URL.

   3

   Replace <ignore_proxy_domains> with a comma separated list of domains. These domains are ignored by the proxy server.

3. Save your changes and quit the text editor to apply your changes.

Verification

1. Verify that the cert-manager controller pod is redeployed by running the following command:

   $ oc get pods -l app.kubernetes.io/name=cert-manager -n cert-manager

   Example output

   NAME                          READY   STATUS    RESTARTS   AGE
   cert-manager-bd7fbb9fc-wvbbt  1/1     Running   0          39s

2. Verify that environment variables are updated for the cert-manager pod by running the following command:

   $ oc get pod <redeployed_cert-manager_controller_pod> -n cert-manager -o yaml

   Example output

       env:
       ...
       - name: HTTP_PROXY
         value: http://<PROXY_URL>
       - name: HTTPS_PROXY
         value: https://<PROXY_URL>
       - name: NO_PROXY
         value: <IGNORE_PROXY_DOMAINS>

Procedure

1. Edit the

   CertManager

   resource by running the following command:

   $ oc edit certmanager cluster

2. Add a

   spec.controllerConfig

   section with the following override arguments:

   apiVersion: operator.openshift.io/v1alpha1
   kind: CertManager
   metadata:
     name: cluster
     ...
   spec:
     ...
     controllerConfig:
       overrideArgs:
         - '--dns01-recursive-nameservers=<host>:<port>' 

   1

   
         - '--dns01-recursive-nameservers-only' 

   2

   
         - '--acme-http01-solver-nameservers=<host>:<port>' 

   3

   
         - '--v=<verbosity_level>' 

   4

   
         - '--metrics-listen-address=<host>:<port>' 

   5

   
         - '--issuer-ambient-credentials' 

   6

   
     webhookConfig:
       overrideArgs:
         - '--v=4' 

   7

   
     cainjectorConfig:
       overrideArgs:
         - '--v=2' 

   8

   1

   Provide a comma-separated list of <host>:<port> nameservers to query for the DNS-01 self check. For example, --dns01-recursive-nameservers=1.1.1.1:53.

   2

   Specify to only use recursive nameservers instead of checking the authoritative nameservers associated with that domain.

   3

   Provide a comma-separated list of <host>:<port> nameservers to query for the Automated Certificate Management Environment (ACME) HTTP01 self check. For example, --acme-http01-solver-nameservers=1.1.1.1:53.

   4 7 8

   Specify to set the log level verbosity to determine the verbosity of log messages.

   5

   Specify the host and port for the metrics endpoint. The default value is --metrics-listen-address=0.0.0.0:9402.

   6

   You must use the --issuer-ambient-credentials argument when configuring an ACME Issuer to solve DNS-01 challenges by using ambient credentials.

3. Save your changes and quit the text editor to apply your changes.

Procedure

1. Check that the

   Certificate

   object and its secret are available by running the following command:

   $ oc get certificate

   Example output

   NAME                                             READY   SECRET                                           AGE
   certificate-from-clusterissuer-route53-ambient   True    certificate-from-clusterissuer-route53-ambient   8h

2. Edit the

   CertManager

   resource by running the following command:

   $ oc edit certmanager cluster

3. Add a

   spec.controllerConfig

   section with the following override arguments:

   apiVersion: operator.openshift.io/v1alpha1
   kind: CertManager
   metadata:
     name: cluster
   # ...
   spec:
   # ...
     controllerConfig:
       overrideArgs:
         - '--enable-certificate-owner-ref'

4. Save your changes and quit the text editor to apply your changes.

Procedure

1. Check that the deployments of the cert-manager controller, CA injector, and Webhook are available by entering the following command:

   $ oc get deployment -n cert-manager

   Example output

   NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
   cert-manager              1/1     1            1           53m
   cert-manager-cainjector   1/1     1            1           53m
   cert-manager-webhook      1/1     1            1           53m

2. Before setting the CPU and memory limit, check the existing configuration for the cert-manager controller, CA injector, and Webhook by entering the following command:

   $ oc get deployment -n cert-manager -o yaml

   Example output

   # ...
     metadata:
       name: cert-manager
       namespace: cert-manager
   # ...
     spec:
       template:
         spec:
           containers:
           - name: cert-manager-controller
             resources: {} 

   1

   
   # ...
     metadata:
       name: cert-manager-cainjector
       namespace: cert-manager
   # ...
     spec:
       template:
         spec:
           containers:
           - name: cert-manager-cainjector
             resources: {} 

   2

   
   # ...
     metadata:
       name: cert-manager-webhook
       namespace: cert-manager
   # ...
     spec:
       template:
         spec:
           containers:
           - name: cert-manager-webhook
             resources: {} 

   3

   
   # ...

   1 2 3

   The spec.resources field is empty by default. The cert-manager components do not have CPU and memory limits.

3. To configure the CPU and memory limits for the cert-manager controller, CA injector, and Webhook, enter the following command:

   $ oc patch certmanager.operator cluster --type=merge -p="
   spec:
     controllerConfig:
       overrideResources:
         limits: 

   1

   
           cpu: 200m 

   2

   
           memory: 64Mi 

   3

   
         requests: 

   4

   
           cpu: 10m 

   5

   
           memory: 16Mi 

   6

   
     webhookConfig:
       overrideResources:
         limits: 

   7

   
           cpu: 200m 

   8

   
           memory: 64Mi 

   9

   
         requests: 

   10

   
           cpu: 10m 

   11

   
           memory: 16Mi 

   12

   
     cainjectorConfig:
       overrideResources:
         limits: 

   13

   
           cpu: 200m 

   14

   
           memory: 64Mi 

   15

   
         requests: 

   16

   
           cpu: 10m 

   17

   
           memory: 16Mi 

   18

   
   "

   1

   Defines the maximum amount of CPU and memory that a single container in a cert-manager controller pod can request.

   2 5

   You can specify the CPU limit that a cert-manager controller pod can request. The default value is 10m.

   3 6

   You can specify the memory limit that a cert-manager controller pod can request. The default value is 32Mi.

   4

   Defines the amount of CPU and memory set by scheduler for the cert-manager controller pod.

   7

   Defines the maximum amount of CPU and memory that a single container in a CA injector pod can request.

   8 11

   You can specify the CPU limit that a CA injector pod can request. The default value is 10m.

   9 12

   You can specify the memory limit that a CA injector pod can request. The default value is 32Mi.

   10

   Defines the amount of CPU and memory set by scheduler for the CA injector pod.

   13

   Defines the maximum amount of CPU and memory Defines the maximum amount of CPU and memory that a single container in a Webhook pod can request.

   14 17

   You can specify the CPU limit that a Webhook pod can request. The default value is 10m.

   15 18

   You can specify the memory limit that a Webhook pod can request. The default value is 32Mi.

   16

   Defines the amount of CPU and memory set by scheduler for the Webhook pod.

   Example output

   certmanager.operator.openshift.io/cluster patched

Verification

1. Verify that the CPU and memory limits are updated for the cert-manager components:

   $ oc get deployment -n cert-manager -o yaml

   Example output

   # ...
     metadata:
       name: cert-manager
       namespace: cert-manager
   # ...
     spec:
       template:
         spec:
           containers:
           - name: cert-manager-controller
             resources:
               limits:
                 cpu: 200m
                 memory: 64Mi
               requests:
                 cpu: 10m
                 memory: 16Mi
   # ...
     metadata:
       name: cert-manager-cainjector
       namespace: cert-manager
   # ...
     spec:
       template:
         spec:
           containers:
           - name: cert-manager-cainjector
             resources:
               limits:
                 cpu: 200m
                 memory: 64Mi
               requests:
                 cpu: 10m
                 memory: 16Mi
   # ...
     metadata:
       name: cert-manager-webhook
       namespace: cert-manager
   # ...
     spec:
       template:
         spec:
           containers:
           - name: cert-manager-webhook
             resources:
               limits:
                 cpu: 200m
                 memory: 64Mi
               requests:
                 cpu: 10m
                 memory: 16Mi
   # ...

Verification

1. Verify pod scheduling settings for

   cert-manager

   pods:

   1. Check the deployments in the

      cert-manager

      namespace to confirm they have the correct

      nodeSelector

      and

      tolerations

      by running the following command:

      $ oc get pods -n cert-manager -o wide

      Example output

      NAME                                       READY   STATUS    RESTARTS   AGE   IP            NODE                         NOMINATED NODE   READINESS GATES
      cert-manager-58d9c69db4-78mzp              1/1     Running   0          10m   10.129.0.36   ip-10-0-1-106.ec2.internal   <none>           <none>
      cert-manager-cainjector-85b6987c66-rhzf7   1/1     Running   0          11m   10.128.0.39   ip-10-0-1-136.ec2.internal   <none>           <none>
      cert-manager-webhook-7f54b4b858-29bsp      1/1     Running   0          11m   10.129.0.35   ip-10-0-1-106.ec2.internal   <none>           <none>

   2. Check the

      nodeSelector

      and

      tolerations

      settings applied to deployments by running the following command:

      $ oc get deployments -n cert-manager -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{.spec.template.spec.nodeSelector}{"\n"}{.spec.template.spec.tolerations}{"\n\n"}{end}'

      Example output

      cert-manager
      {"kubernetes.io/os":"linux","node-role.kubernetes.io/control-plane":""}
      [{"effect":"NoSchedule","key":"node-role.kubernetes.io/master","operator":"Exists"}]
      
      cert-manager-cainjector
      {"kubernetes.io/os":"linux","node-role.kubernetes.io/control-plane":""}
      [{"effect":"NoSchedule","key":"node-role.kubernetes.io/master","operator":"Exists"}]
      
      cert-manager-webhook
      {"kubernetes.io/os":"linux","node-role.kubernetes.io/control-plane":""}
      [{"effect":"NoSchedule","key":"node-role.kubernetes.io/master","operator":"Exists"}]

2. Verify pod scheduling events in the

   cert-manager

   namespace by running the following command:

   $ oc get events -n cert-manager --field-selector reason=Scheduled

Procedure

1. Create a directory to store a

   CredentialsRequest

   resource YAML file by running the following command:

   $ mkdir credentials-request

2. Create a

   CredentialsRequest

   resource YAML file under the

   credentials-request

   directory, such as,

   sample-credential-request.yaml

   , by applying the following yaml:

   apiVersion: cloudcredential.openshift.io/v1
   kind: CredentialsRequest
   metadata:
     name: cert-manager
     namespace: openshift-cloud-credential-operator
   spec:
     providerSpec:
       apiVersion: cloudcredential.openshift.io/v1
       kind: AWSProviderSpec
       statementEntries:
       - action:
         - "route53:GetChange"
         effect: Allow
         resource: "arn:aws:route53:::change/*"
       - action:
         - "route53:ChangeResourceRecordSets"
         - "route53:ListResourceRecordSets"
         effect: Allow
         resource: "arn:aws:route53:::hostedzone/*"
       - action:
         - "route53:ListHostedZonesByName"
         effect: Allow
         resource: "*"
     secretRef:
       name: aws-creds
       namespace: cert-manager
     serviceAccountNames:
     - cert-manager

3. Use the

   ccoctl

   tool to process

   CredentialsRequest

   objects by running the following command:

   $ ccoctl aws create-iam-roles \
       --name <user_defined_name> --region=<aws_region> \
       --credentials-requests-dir=<path_to_credrequests_dir> \
       --identity-provider-arn <oidc_provider_arn> --output-dir=<path_to_output_dir>

   Example output

   2023/05/15 18:10:34 Role arn:aws:iam::XXXXXXXXXXXX:role/<user_defined_name>-cert-manager-aws-creds created
   2023/05/15 18:10:34 Saved credentials configuration to: <path_to_output_dir>/manifests/cert-manager-aws-creds-credentials.yaml
   2023/05/15 18:10:35 Updated Role policy for Role <user_defined_name>-cert-manager-aws-creds

   Copy the

   <aws_role_arn>

   from the output to use in the next step. For example,

   "arn:aws:iam::XXXXXXXXXXXX:role/<user_defined_name>-cert-manager-aws-creds"

4. Add the

   eks.amazonaws.com/role-arn="<aws_role_arn>"

   annotation to the service account by running the following command:

   $ oc -n cert-manager annotate serviceaccount cert-manager eks.amazonaws.com/role-arn="<aws_role_arn>"

5. To create a new pod, delete the existing cert-manager controller pod by running the following command:

   $ oc delete pods -l app.kubernetes.io/name=cert-manager -n cert-manager

   The AWS credentials are applied to a new cert-manager controller pod within a minute.

Verification

1. Get the name of the updated cert-manager controller pod by running the following command:

   $ oc get pods -l app.kubernetes.io/name=cert-manager -n cert-manager

   Example output

   NAME                          READY   STATUS    RESTARTS   AGE
   cert-manager-bd7fbb9fc-wvbbt  1/1     Running   0          39s

2. Verify that AWS credentials are updated by running the following command:

   $ oc set env -n cert-manager po/<cert_manager_controller_pod_name> --list

   Example output

   # pods/cert-manager-57f9555c54-vbcpg, container cert-manager-controller
   # POD_NAMESPACE from field path metadata.namespace
   AWS_ROLE_ARN=XXXXXXXXXXXX
   AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token

Procedure

1. Edit the

   CertManager

   resource by running the following command:

   $ oc edit certmanager.operator cluster

2. Set the log level value by editing the

   spec.logLevel

   section:

   apiVersion: operator.openshift.io/v1alpha1
   kind: CertManager
   ...
   spec:
     logLevel: Normal 

   1

   1

   The default logLevel is Normal. Replace Normal with the desired log level value. The valid log level values for the CertManager resource are Normal, Debug, Trace, and TraceAll. To audit logs and perform common operations when everything is fine, set logLevel to Normal . To troubleshoot a minor issue by viewing verbose logs, set logLevel to Debug . To troubleshoot a major issue by viewing more verbose logs, you can set logLevel to Trace. To troubleshoot serious issues, set logLevel to TraceAll.

   Note

   TraceAll

   generates huge amount of logs. After setting

   logLevel

   to

   TraceAll

   , you might experience performance issues.

3. Save your changes and quit the text editor to apply your changes.

   After applying the changes, the verbosity level for the cert-manager components controller, CA injector, and webhook is updated.

Verification

1. The cert-manager Operator pod is redeployed. Verify that the log level of the cert-manager Operator for Red Hat OpenShift is updated by running the following command:

   $ oc set env deploy/cert-manager-operator-controller-manager -n cert-manager-operator --list | grep -e OPERATOR_LOG_LEVEL -e container

   Example output

   # deployments/cert-manager-operator-controller-manager, container kube-rbac-proxy
   OPERATOR_LOG_LEVEL=9
   # deployments/cert-manager-operator-controller-manager, container cert-manager-operator
   OPERATOR_LOG_LEVEL=9

2. Verify that the log level of the cert-manager Operator for Red Hat OpenShift is updated by running the

   oc logs

   command:

   $ oc logs deploy/cert-manager-operator-controller-manager -n cert-manager-operator