Chapter 9. Installing a private cluster on GCP

1. One vCPU is equivalent to one physical core when simultaneous multithreading (SMT), or Hyper-Threading, is not enabled. When enabled, use the following formula to calculate the corresponding ratio: (threads per core × cores) × sockets = vCPUs.
2. OpenShift Container Platform and Kubernetes are sensitive to disk performance, and faster storage is recommended, particularly for etcd on the control plane nodes which require a 10 ms p99 fsync duration. Note that on many cloud platforms, storage size and IOPS scale together, so you might need to over-allocate storage volume to obtain sufficient performance.
3. As with all user-provisioned installations, if you choose to use RHEL compute machines in your cluster, you take responsibility for all operating system life cycle management and maintenance, including performing system updates, applying patches, and completing all other required tasks. Use of RHEL 7 compute machines is deprecated and has been removed in OpenShift Container Platform 4.10 and later.

1. Change to the directory that contains the installation program and create a manifest file:

   $ ./openshift-install create manifests --dir <installation_directory> 1

   1

   For <installation_directory>, specify the name of the directory that contains the install-config.yaml file for your cluster.

2. Create a file that is named cluster-ingress-default-ingresscontroller.yaml in the <installation_directory>/manifests/ directory:

   $ touch <installation_directory>/manifests/cluster-ingress-default-ingresscontroller.yaml 1

   1

   For <installation_directory>, specify the directory name that contains the manifests/ directory for your cluster.

   After creating the file, several network configuration files are in the manifests/ directory, as shown:

   $ ls <installation_directory>/manifests/cluster-ingress-default-ingresscontroller.yaml

   Example output

   cluster-ingress-default-ingresscontroller.yaml

3. Open the cluster-ingress-default-ingresscontroller.yaml file in an editor and enter a custom resource (CR) that describes the Operator configuration you want:

   Sample clientAccess configuration to Global

     apiVersion: operator.openshift.io/v1
     kind: IngressController
     metadata:
       name: default
       namespace: openshift-ingress-operator
     spec:
       endpointPublishingStrategy:
         loadBalancer:
           providerParameters:
             gcp:
               clientAccess: Global 1
             type: GCP
           scope: Internal          2
         type: LoadBalancerService

   1

   Set gcp.clientAccess to Global.

   2

   Global access is only available to Ingress Controllers using internal load balancers.

Procedure

1. Edit your install-config.yaml file and add the proxy settings. For example:

   apiVersion: v1
   baseDomain: my.domain.com
   proxy:
     httpProxy: http://<username>:<pswd>@<ip>:<port> 1
     httpsProxy: https://<username>:<pswd>@<ip>:<port> 2
     noProxy: example.com 3
   additionalTrustBundle: | 4
       -----BEGIN CERTIFICATE-----
       <MY_TRUSTED_CA_CERT>
       -----END CERTIFICATE-----
   additionalTrustBundlePolicy: <policy_to_add_additionalTrustBundle> 5

   1

   A proxy URL to use for creating HTTP connections outside the cluster. The URL scheme must be http.

   2

   A proxy URL to use for creating HTTPS connections outside the cluster.

   3

   A comma-separated list of destination domain names, IP addresses, or other network CIDRs to exclude from proxying. Preface a domain with . to match subdomains only. For example, .y.com matches x.y.com, but not y.com. Use * to bypass the proxy for all destinations.

   4

   If provided, the installation program generates a config map that is named user-ca-bundle in the openshift-config namespace that contains one or more additional CA certificates that are required for proxying HTTPS connections. The Cluster Network Operator then creates a trusted-ca-bundle config map that merges these contents with the Red Hat Enterprise Linux CoreOS (RHCOS) trust bundle, and this config map is referenced in the trustedCA field of the Proxy object. The additionalTrustBundle field is required unless the proxy’s identity certificate is signed by an authority from the RHCOS trust bundle.

   5

   Optional: The policy to determine the configuration of the Proxy object to reference the user-ca-bundle config map in the trustedCA field. The allowed values are Proxyonly and Always. Use Proxyonly to reference the user-ca-bundle config map only when http/https proxy is configured. Use Always to always reference the user-ca-bundle config map. The default value is Proxyonly.

   Note

   The installation program does not support the proxy readinessEndpoints field.

   Note

   If the installer times out, restart and then complete the deployment by using the wait-for command of the installer. For example:

   $ ./openshift-install wait-for install-complete --log-level debug

2. Save the file and reference it when installing OpenShift Container Platform.

Procedure

1. Add the following granular permissions to the GCP account that the installation program uses:

   Example 9.3. Required GCP permissions

   • compute.machineTypes.list
   • compute.regions.list
   • compute.zones.list
   • dns.changes.create
   • dns.changes.get
   • dns.managedZones.create
   • dns.managedZones.delete
   • dns.managedZones.get
   • dns.managedZones.list
   • dns.networks.bindPrivateDNSZone
   • dns.resourceRecordSets.create
   • dns.resourceRecordSets.delete
   • dns.resourceRecordSets.list

2. If you did not set the credentialsMode parameter in the install-config.yaml configuration file to Manual, modify the value as shown:

   Sample configuration file snippet

   apiVersion: v1
   baseDomain: example.com
   credentialsMode: Manual
   # ...

3. If you have not previously created installation manifest files, do so by running the following command:

   $ openshift-install create manifests --dir <installation_directory>

   where <installation_directory> is the directory in which the installation program creates files.

4. Set a $RELEASE_IMAGE variable with the release image from your installation file by running the following command:

   $ RELEASE_IMAGE=$(./openshift-install version | awk '/release image/ {print $3}')

5. Extract the list of CredentialsRequest custom resources (CRs) from the OpenShift Container Platform release image by running the following command:

   $ oc adm release extract \
     --from=$RELEASE_IMAGE \
     --credentials-requests \
     --included \1
     --install-config=<path_to_directory_with_installation_configuration>/install-config.yaml \2
     --to=<path_to_directory_for_credentials_requests> 3

   1

   The --included parameter includes only the manifests that your specific cluster configuration requires.

   2

   Specify the location of the install-config.yaml file.

   3

   Specify the path to the directory where you want to store the CredentialsRequest objects. If the specified directory does not exist, this command creates it.

   This command creates a YAML file for each CredentialsRequest object.

   Sample CredentialsRequest object

   apiVersion: cloudcredential.openshift.io/v1
   kind: CredentialsRequest
   metadata:
     name: <component_credentials_request>
     namespace: openshift-cloud-credential-operator
     ...
   spec:
     providerSpec:
       apiVersion: cloudcredential.openshift.io/v1
       kind: GCPProviderSpec
       predefinedRoles:
       - roles/storage.admin
       - roles/iam.serviceAccountUser
       skipServiceCheck: true
     ...

6. Create YAML files for secrets in the openshift-install manifests directory that you generated previously. The secrets must be stored using the namespace and secret name defined in the spec.secretRef for each CredentialsRequest object.

   Sample CredentialsRequest object with secrets

   apiVersion: cloudcredential.openshift.io/v1
   kind: CredentialsRequest
   metadata:
     name: <component_credentials_request>
     namespace: openshift-cloud-credential-operator
     ...
   spec:
     providerSpec:
       apiVersion: cloudcredential.openshift.io/v1
         ...
     secretRef:
       name: <component_secret>
       namespace: <component_namespace>
     ...

   Sample Secret object

   apiVersion: v1
   kind: Secret
   metadata:
     name: <component_secret>
     namespace: <component_namespace>
   data:
     service_account.json: <base64_encoded_gcp_service_account_file>