Installing

1. The bootstrap machine boots and starts hosting the remote resources required for the control plane machines to boot. (Requires manual intervention if you provision the infrastructure)
2. The bootstrap machine starts a single-node etcd cluster and a temporary Kubernetes control plane.
3. The control plane machines fetch the remote resources from the bootstrap machine and finish booting. (Requires manual intervention if you provision the infrastructure)
4. The temporary control plane schedules the production control plane to the production control plane machines.
5. The Cluster Version Operator (CVO) comes online and installs the etcd Operator. The etcd Operator scales up etcd on all control plane nodes.
6. The temporary control plane shuts down and passes control to the production control plane.
7. The bootstrap machine injects OpenShift Container Platform components into the production control plane.
8. The installation program shuts down the bootstrap machine. (Requires manual intervention if you provision the infrastructure)
9. The control plane sets up the compute nodes.
10. The control plane installs additional services in the form of a set of Operators.

Procedure

1. Show the status of all worker nodes:

   $ oc get nodes

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                           STATUS   ROLES    AGE   VERSION
   example-compute1.example.com   Ready    worker   13m   v1.21.6+bb8d50a
   example-compute2.example.com   Ready    worker   13m   v1.21.6+bb8d50a
   example-compute4.example.com   Ready    worker   14m   v1.21.6+bb8d50a
   example-control1.example.com   Ready    master   52m   v1.21.6+bb8d50a
   example-control2.example.com   Ready    master   55m   v1.21.6+bb8d50a
   example-control3.example.com   Ready    master   55m   v1.21.6+bb8d50a

   Copy to Clipboard Toggle word wrap

2. Show the phase of all worker machine nodes:

   $ oc get machines -A

   Copy to Clipboard Toggle word wrap

   Example output

   NAMESPACE               NAME                           PHASE         TYPE   REGION   ZONE   AGE
   openshift-machine-api   example-zbbt6-master-0         Running                              95m
   openshift-machine-api   example-zbbt6-master-1         Running                              95m
   openshift-machine-api   example-zbbt6-master-2         Running                              95m
   openshift-machine-api   example-zbbt6-worker-0-25bhp   Running                              49m
   openshift-machine-api   example-zbbt6-worker-0-8b4c2   Running                              49m
   openshift-machine-api   example-zbbt6-worker-0-jkbqt   Running                              49m
   openshift-machine-api   example-zbbt6-worker-0-qrl5b   Running                              49m

   Copy to Clipboard Toggle word wrap

Procedure

1. Download the mirror-registry.tar.gz package for the latest version of the mirror registry for Red Hat OpenShift found on the OpenShift console Downloads page.

2. Install the mirror registry for Red Hat OpenShift on your local host with your current user account by using the mirror-registry tool. For a full list of available flags, see "mirror registry for Red Hat OpenShift flags".

   $ ./mirror-registry install \
     --quayHostname <host_example_com> \
     --quayRoot <example_directory_name>

   Copy to Clipboard Toggle word wrap

3. Use the user name and password generated during installation to log into the registry by running the following command:

   $ podman login -u init \
     -p <password> \
     <host_example_com>:8443> \
     --tls-verify=false 

   1

   Copy to Clipboard Toggle word wrap

   1

   You can avoid running --tls-verify=false by configuring your system to trust the generated rootCA certificates. See "Using SSL to protect connections to Red Hat Quay" and "Configuring the system to trust the certificate authority" for more information.

   Note

   You can also log in by accessing the UI at https://<host.example.com>:8443 after installation.

4. You can mirror OpenShift Container Platform images after logging in. Depending on your needs, see either the "Mirroring the OpenShift Container Platform image repository" or the "Mirroring Operator catalogs for use with disconnected clusters" sections of this document.

   Note

   If there are issues with images stored by the mirror registry for Red Hat OpenShift due to storage layer problems, you can remirror the OpenShift Container Platform images, or reinstall mirror registry on more stable storage.

Procedure

1. Download the mirror-registry.tar.gz package for the latest version of the mirror registry for Red Hat OpenShift found on the OpenShift console Downloads page.

2. Install the mirror registry for Red Hat OpenShift on your local host with your current user account by using the mirror-registry tool. For a full list of available flags, see "mirror registry for Red Hat OpenShift flags".

   $ ./mirror-registry install -v \
     --targetHostname <host_example_com> \
     --targetUsername <example_user> \
     -k ~/.ssh/my_ssh_key \
     --quayHostname <host_example_com> \
     --quayRoot <example_directory_name>

   Copy to Clipboard Toggle word wrap

3. Use the user name and password generated during installation to log into the mirror registry by running the following command:

   $ podman login -u init \
     -p <password> \
     <host_example_com>:8443> \
     --tls-verify=false 

   1

   Copy to Clipboard Toggle word wrap

   1

   You can avoid running --tls-verify=false by configuring your system to trust the generated rootCA certificates. See "Using SSL to protect connections to Red Hat Quay" and "Configuring the system to trust the certificate authority" for more information.

   Note

   You can also log in by accessing the UI at https://<host.example.com>:8443 after installation.

4. You can mirror OpenShift Container Platform images after logging in. Depending on your needs, see either the "Mirroring the OpenShift Container Platform image repository" or the "Mirroring Operator catalogs for use with disconnected clusters" sections of this document.

   Note

   If there are issues with images stored by the mirror registry for Red Hat OpenShift due to storage layer problems, you can remirror the OpenShift Container Platform images, or reinstall mirror registry on more stable storage.

1. Download your registry.redhat.io pull secret from the Red Hat OpenShift Cluster Manager.

2. Make a copy of your pull secret in JSON format:

   $ cat ./pull-secret | jq . > <path>/<pull_secret_file_in_json> 

   1

   Copy to Clipboard Toggle word wrap

   1

   Specify the path to the folder to store the pull secret in and a name for the JSON file that you create.

   The contents of the file resemble the following example:

   {
     "auths": {
       "cloud.openshift.com": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "quay.io": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "registry.connect.redhat.com": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       },
       "registry.redhat.io": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       }
     }
   }

   Copy to Clipboard Toggle word wrap

3. Generate the base64-encoded user name and password or token for your mirror registry:

   $ echo -n '<user_name>:<password>' | base64 -w0 

   1

   
   BGVtbYk3ZHAtqXs=

   Copy to Clipboard Toggle word wrap

   1

   For <user_name> and <password>, specify the user name and password that you configured for your registry.

4. Edit the JSON file and add a section that describes your registry to it:

     "auths": {
       "<mirror_registry>": { 

   1

   
         "auth": "<credentials>", 

   2

   
         "email": "you@example.com"
       }
     },

   Copy to Clipboard Toggle word wrap

   1

   For <mirror_registry>, specify the registry domain name, and optionally the port, that your mirror registry uses to serve content. For example, registry.example.com or registry.example.com:8443

   2

   For <credentials>, specify the base64-encoded user name and password for the mirror registry.

   The file resembles the following example:

   {
     "auths": {
       "registry.example.com": {
         "auth": "BGVtbYk3ZHAtqXs=",
         "email": "you@example.com"
       },
       "cloud.openshift.com": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "quay.io": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "registry.connect.redhat.com": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       },
       "registry.redhat.io": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       }
     }
   }

   Copy to Clipboard Toggle word wrap

1. Review the OpenShift Container Platform downloads page to determine the version of OpenShift Container Platform that you want to install and determine the corresponding tag on the Repository Tags page.

2. Set the required environment variables:

   1. Export the release version:

      $ OCP_RELEASE=<release_version>

      Copy to Clipboard Toggle word wrap

      For <release_version>, specify the tag that corresponds to the version of OpenShift Container Platform to install, such as 4.5.4.

   2. Export the local registry name and host port:

      $ LOCAL_REGISTRY='<local_registry_host_name>:<local_registry_host_port>'

      Copy to Clipboard Toggle word wrap

      For <local_registry_host_name>, specify the registry domain name for your mirror repository, and for <local_registry_host_port>, specify the port that it serves content on.

   3. Export the local repository name:

      $ LOCAL_REPOSITORY='<local_repository_name>'

      Copy to Clipboard Toggle word wrap

      For <local_repository_name>, specify the name of the repository to create in your registry, such as ocp4/openshift4.

   4. Export the name of the repository to mirror:

      $ PRODUCT_REPO='openshift-release-dev'

      Copy to Clipboard Toggle word wrap

      For a production release, you must specify openshift-release-dev.

   5. Export the path to your registry pull secret:

      $ LOCAL_SECRET_JSON='<path_to_pull_secret>'

      Copy to Clipboard Toggle word wrap

      For <path_to_pull_secret>, specify the absolute path to and file name of the pull secret for your mirror registry that you created.

   6. Export the release mirror:

      $ RELEASE_NAME="ocp-release"

      Copy to Clipboard Toggle word wrap

      For a production release, you must specify ocp-release.

   7. Export the type of architecture for your server, such as x86_64:

      $ ARCHITECTURE=<server_architecture>

      Copy to Clipboard Toggle word wrap

   8. Export the path to the directory to host the mirrored images:

      $ REMOVABLE_MEDIA_PATH=<path> 

      1

      Copy to Clipboard Toggle word wrap

      1

      Specify the full path, including the initial forward slash (/) character.

3. Mirror the version images to the mirror registry:

   • If your mirror host does not have internet access, take the following actions:

     1. Connect the removable media to a system that is connected to the internet.

     2. Review the images and configuration manifests to mirror:

        $ oc adm release mirror -a ${LOCAL_SECRET_JSON}  \
             --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE} \
             --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
             --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE} --dry-run

        Copy to Clipboard Toggle word wrap
     3. Record the entire imageContentSources section from the output of the previous command. The information about your mirrors is unique to your mirrored repository, and you must add the imageContentSources section to the install-config.yaml file during installation.

     4. Mirror the images to a directory on the removable media:

        $ oc adm release mirror -a ${LOCAL_SECRET_JSON} --to-dir=${REMOVABLE_MEDIA_PATH}/mirror quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE}

        Copy to Clipboard Toggle word wrap

     5. Take the media to the restricted network environment and upload the images to the local container registry.

        $ oc image mirror -a ${LOCAL_SECRET_JSON} --from-dir=${REMOVABLE_MEDIA_PATH}/mirror "file://openshift/release:${OCP_RELEASE}*" ${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} 

        1

        Copy to Clipboard Toggle word wrap

        1

        For REMOVABLE_MEDIA_PATH, you must use the same path that you specified when you mirrored the images.

        Important

        Running oc image mirror might result in the following error: error: unable to retrieve source image. This error occurs when image indexes include references to images that no longer exist on the image registry. Image indexes might retain older references to allow users running those images an upgrade path to newer points on the upgrade graph. As a temporary workaround, you can use the --skip-missing option to bypass the error and continue downloading the image index. For more information, see Service Mesh Operator mirroring failed.

   • If the local container registry is connected to the mirror host, take the following actions:

     1. Directly push the release images to the local registry by using following command:

        $ oc adm release mirror -a ${LOCAL_SECRET_JSON}  \
             --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE} \
             --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
             --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}

        Copy to Clipboard Toggle word wrap

        This command pulls the release information as a digest, and its output includes the imageContentSources data that you require when you install your cluster.

     2. Record the entire imageContentSources section from the output of the previous command. The information about your mirrors is unique to your mirrored repository, and you must add the imageContentSources section to the install-config.yaml file during installation.

        Note

        The image name gets patched to Quay.io during the mirroring process, and the podman images will show Quay.io in the registry on the bootstrap virtual machine.

4. To create the installation program that is based on the content that you mirrored, extract it and pin it to the release:

   • If your mirror host does not have internet access, run the following command:

     $ oc adm release extract -a ${LOCAL_SECRET_JSON} --icsp-file=<file> \ --command=openshift-install "${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}"

     Copy to Clipboard Toggle word wrap

   • If the local container registry is connected to the mirror host, run the following command:

     $ oc adm release extract -a ${LOCAL_SECRET_JSON} --command=openshift-install "${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}"

     Copy to Clipboard Toggle word wrap
     Important

     To ensure that you use the correct images for the version of OpenShift Container Platform that you selected, you must extract the installation program from the mirrored content.

     You must perform this step on a machine with an active internet connection.

5. For clusters using installer-provisioned infrastructure, run the following command:

   $ openshift-install

   Copy to Clipboard Toggle word wrap

Procedure

1. Create an ImageSetConfiguration resource that specifies the necessary configuration details:

   Example ImageSetConfiguration file

   apiVersion: mirror.openshift.io/v1alpha1
   kind: ImageSetConfiguration
   archiveSize: 4 

   1

   
   mirror:
    ocp:
      channels:
        - name: stable-4.9 

   2

   
    operators:
      - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.9 

   3

   
   storageConfig: 

   4

   
    registry:
      imageURL: example.com/example/oc-mirror 

   5

   Copy to Clipboard Toggle word wrap

   1

   The maximum size, in GiB, of each file within the image set.

   2

   The channel to retrieve the OpenShift Container Platform images from.

   3

   The Operator catalog to retrieve the OpenShift Container Platform images from.

   4

   The back-end location to save the image set metadata to. This location can be a registry or local directory. It is required to specify storageConfig values.

   5

   The registry URL for the storage backend.

   This example pulls images from the stable-4.9 channel for the registry.redhat.io/redhat/redhat-operator-index:v4.9 operator catalog and saves the image set metadata to the example.com/example/oc-mirror registry.

2. Save the file as imageset-config.yaml. This file is required by the oc mirror command when mirroring content.

Procedure

1. Log in to the OpenShift CLI as a user with the cluster-admin role.

2. Apply the YAML files from the results directory to the cluster by running the following command:

   $ oc apply -f ./oc-mirror-workspace/results-1639608409/

   Copy to Clipboard Toggle word wrap

3. Apply the release image signatures to the cluster by running the following command:

   $ oc apply -f ./oc-mirror-workspace/results-1639608409/release-signatures/

   Copy to Clipboard Toggle word wrap

Verification

1. Verify that the ImageContentSourcePolicy resources were successfully installed by running the following command:

   $ oc get imagecontentsourcepolicy --all-namespaces

   Copy to Clipboard Toggle word wrap

2. Verify that the CatalogSource resources were successfully installed by running the following command:

   $ oc get catalogsource --all-namespaces

   Copy to Clipboard Toggle word wrap

Procedure

1. Follow the same steps that you used to create the initial image set and mirror it to the mirror registry. For instructions, see Mirroring an image set in a partially disconnected environment or Mirroring an image set in a fully disconnected environment.

   Important
   • You must provide the same storage backend so that only a differential image set is created and mirrored.
   • If you specified a top-level namespace for the mirror registry during the initial image set creation, then you must use this same namespace every time you run the oc-mirror plugin for the same mirror registry.
2. Configure your cluster to use the resources generated by oc-mirror.

Procedure

1. Identify your domain, or subdomain, and registrar. You can transfer an existing domain and registrar or obtain a new one through Alibaba Cloud or another source.

   Note

   If you purchase a new domain through Alibaba Cloud, it takes time for the relevant DNS changes to propagate. For more information about purchasing domains through Alibaba Cloud, see Alibaba Cloud domains.

2. If you are using an existing domain and registrar, migrate its DNS to Alibaba Cloud. See Domain name transfer in the Alibaba Cloud documentation.

3. Configure DNS for your domain. This includes:

   • Registering a generic domain name.
   • Completing real-name verification for your domain name.
   • Applying for an Internet Content Provider (ICP) filing.

   • Enabling domain name resolution.

     Use an appropriate root domain, such as openshiftcorp.com, or subdomain, such as clusters.openshiftcorp.com.

4. If you are using a subdomain, follow the procedures of your company to add its delegation records to the parent domain.

Procedure

1. Obtain the OpenShift Container Platform release image:

   $ RELEASE_IMAGE=$(./openshift-install version | awk '/release image/ {print $3}')

   Copy to Clipboard Toggle word wrap

2. Get the CCO container image from the OpenShift Container Platform release image:

   $ CCO_IMAGE=$(oc adm release info --image-for='cloud-credential-operator' $RELEASE_IMAGE)

   Copy to Clipboard Toggle word wrap
   Note

   Ensure that the architecture of the $RELEASE_IMAGE matches the architecture of the environment in which you will use the ccoctl tool.

3. Extract the ccoctl binary from the CCO container image within the OpenShift Container Platform release image:

   $ oc image extract $CCO_IMAGE --file="/usr/bin/ccoctl" -a ~/.pull-secret

   Copy to Clipboard Toggle word wrap

4. Change the permissions to make ccoctl executable:

   $ chmod 775 ccoctl

   Copy to Clipboard Toggle word wrap