Disconnected installation mirroring

Procedure

1. Download the mirror-registry.tar.gz package for the latest version of the mirror registry for Red Hat OpenShift found on the OpenShift console Downloads page.

2. Install the mirror registry for Red Hat OpenShift on your local host with your current user account by using the mirror-registry tool. For a full list of available flags, see "mirror registry for Red Hat OpenShift flags".

   $ ./mirror-registry install \
     --quayHostname <host_example_com> \
     --quayRoot <example_directory_name>

3. Use the user name and password generated during installation to log into the registry by running the following command:

   $ podman login -u init \
     -p <password> \
     <host_example_com>:8443> \
     --tls-verify=false 1

   1

   You can avoid running --tls-verify=false by configuring your system to trust the generated rootCA certificates. See "Using SSL to protect connections to Red Hat Quay" and "Configuring the system to trust the certificate authority" for more information.

   Note

   You can also log in by accessing the UI at https://<host.example.com>:8443 after installation.

4. You can mirror OpenShift Container Platform images after logging in. Depending on your needs, see either the "Mirroring the OpenShift Container Platform image repository" or the "Mirroring Operator catalogs for use with disconnected clusters" sections of this document.

   Note

   If there are issues with images stored by the mirror registry for Red Hat OpenShift due to storage layer problems, you can remirror the OpenShift Container Platform images, or reinstall mirror registry on more stable storage.

Procedure

1. Download the mirror-registry.tar.gz package for the latest version of the mirror registry for Red Hat OpenShift found on the OpenShift console Downloads page.

2. Install the mirror registry for Red Hat OpenShift on your local host with your current user account by using the mirror-registry tool. For a full list of available flags, see "mirror registry for Red Hat OpenShift flags".

   $ ./mirror-registry install -v \
     --targetHostname <host_example_com> \
     --targetUsername <example_user> \
     -k ~/.ssh/my_ssh_key \
     --quayHostname <host_example_com> \
     --quayRoot <example_directory_name>

3. Use the user name and password generated during installation to log into the mirror registry by running the following command:

   $ podman login -u init \
     -p <password> \
     <host_example_com>:8443> \
     --tls-verify=false 1

   1

   You can avoid running --tls-verify=false by configuring your system to trust the generated rootCA certificates. See "Using SSL to protect connections to Red Hat Quay" and "Configuring the system to trust the certificate authority" for more information.

   Note

   You can also log in by accessing the UI at https://<host.example.com>:8443 after installation.

4. You can mirror OpenShift Container Platform images after logging in. Depending on your needs, see either the "Mirroring the OpenShift Container Platform image repository" or the "Mirroring Operator catalogs for use with disconnected clusters" sections of this document.

   Note

   If there are issues with images stored by the mirror registry for Red Hat OpenShift due to storage layer problems, you can remirror the OpenShift Container Platform images, or reinstall mirror registry on more stable storage.

Procedure

1. Enter the following command to install the mirror registry for Red Hat OpenShift:

   $ ./mirror-registry install \
   --quayHostname <host_example_com> \
   --quayRoot <example_directory_name>

   This installs the mirror registry for Red Hat OpenShift to the $HOME/quay-install directory.

2. Prepare a new certificate authority (CA) bundle and generate new ssl.key and ssl.crt key files. For more information, see Using SSL/TLS to protect connections to Red Hat Quay.

3. Assign /$HOME/quay-install an environment variable, for example, QUAY, by entering the following command:

   $ export QUAY=/$HOME/quay-install

4. Copy the new ssl.crt file to the /$HOME/quay-install directory by entering the following command:

   $ cp ~/ssl.crt $QUAY/quay-config

5. Copy the new ssl.key file to the /$HOME/quay-install directory by entering the following command:

   $ cp ~/ssl.key $QUAY/quay-config

6. Restart the quay-app application pod by entering the following command:

   $ systemctl restart quay-app

1. Download your registry.redhat.io pull secret from Red Hat OpenShift Cluster Manager.

2. Make a copy of your pull secret in JSON format:

   $ cat ./pull-secret | jq . > <path>/<pull_secret_file_in_json> 1

   1

   Specify the path to the folder to store the pull secret in and a name for the JSON file that you create.

   The contents of the file resemble the following example:

   {
     "auths": {
       "cloud.openshift.com": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "quay.io": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "registry.connect.redhat.com": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       },
       "registry.redhat.io": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       }
     }
   }

3. Generate the base64-encoded user name and password or token for your mirror registry:

   $ echo -n '<user_name>:<password>' | base64 -w0 1
   BGVtbYk3ZHAtqXs=

   1

   For <user_name> and <password>, specify the user name and password that you configured for your registry.

4. Edit the JSON file and add a section that describes your registry to it:

     "auths": {
       "<mirror_registry>": { 1
         "auth": "<credentials>", 2
         "email": "you@example.com"
       }
     },

   1

   For <mirror_registry>, specify the registry domain name, and optionally the port, that your mirror registry uses to serve content. For example, registry.example.com or registry.example.com:8443

   2

   For <credentials>, specify the base64-encoded user name and password for the mirror registry.

   The file resembles the following example:

   {
     "auths": {
       "registry.example.com": {
         "auth": "BGVtbYk3ZHAtqXs=",
         "email": "you@example.com"
       },
       "cloud.openshift.com": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "quay.io": {
         "auth": "b3BlbnNo...",
         "email": "you@example.com"
       },
       "registry.connect.redhat.com": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       },
       "registry.redhat.io": {
         "auth": "NTE3Njg5Nj...",
         "email": "you@example.com"
       }
     }
   }

1. Review the OpenShift Container Platform downloads page to determine the version of OpenShift Container Platform that you want to install and determine the corresponding tag on the Repository Tags page.

2. Set the required environment variables:

   1. Export the release version:

      $ OCP_RELEASE=<release_version>

      For <release_version>, specify the tag that corresponds to the version of OpenShift Container Platform to install, such as 4.5.4.

   2. Export the local registry name and host port:

      $ LOCAL_REGISTRY='<local_registry_host_name>:<local_registry_host_port>'

      For <local_registry_host_name>, specify the registry domain name for your mirror repository, and for <local_registry_host_port>, specify the port that it serves content on.

   3. Export the local repository name:

      $ LOCAL_REPOSITORY='<local_repository_name>'

      For <local_repository_name>, specify the name of the repository to create in your registry, such as ocp4/openshift4.

   4. Export the name of the repository to mirror:

      $ PRODUCT_REPO='openshift-release-dev'

      For a production release, you must specify openshift-release-dev.

   5. Export the path to your registry pull secret:

      $ LOCAL_SECRET_JSON='<path_to_pull_secret>'

      For <path_to_pull_secret>, specify the absolute path to and file name of the pull secret for your mirror registry that you created.

   6. Export the release mirror:

      $ RELEASE_NAME="ocp-release"

      For a production release, you must specify ocp-release.

   7. Export the type of architecture for your cluster:

      $ ARCHITECTURE=<cluster_architecture> 1

      1

      Specify the architecture of the cluster, such as x86_64, aarch64, s390x, or ppc64le.

   8. Export the path to the directory to host the mirrored images:

      $ REMOVABLE_MEDIA_PATH=<path> 1

      1

      Specify the full path, including the initial forward slash (/) character.

3. Mirror the version images to the mirror registry:

   • If your mirror host does not have internet access, take the following actions:

     1. Connect the removable media to a system that is connected to the internet.

     2. Review the images and configuration manifests to mirror:

        $ oc adm release mirror -a ${LOCAL_SECRET_JSON}  \
             --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE} \
             --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
             --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE} --dry-run

     3. Record the entire imageContentSources section from the output of the previous command. The information about your mirrors is unique to your mirrored repository, and you must add the imageContentSources section to the install-config.yaml file during installation.

     4. Mirror the images to a directory on the removable media:

        $ oc adm release mirror -a ${LOCAL_SECRET_JSON} --to-dir=${REMOVABLE_MEDIA_PATH}/mirror quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE}

     5. Take the media to the restricted network environment and upload the images to the local container registry.

        $ oc image mirror -a ${LOCAL_SECRET_JSON} --from-dir=${REMOVABLE_MEDIA_PATH}/mirror "file://openshift/release:${OCP_RELEASE}*" ${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} 1

        1

        For REMOVABLE_MEDIA_PATH, you must use the same path that you specified when you mirrored the images.

        Important

        Running oc image mirror might result in the following error: error: unable to retrieve source image. This error occurs when image indexes include references to images that no longer exist on the image registry. Image indexes might retain older references to allow users running those images an upgrade path to newer points on the upgrade graph. As a temporary workaround, you can use the --skip-missing option to bypass the error and continue downloading the image index. For more information, see Service Mesh Operator mirroring failed.

   • If the local container registry is connected to the mirror host, take the following actions:

     1. Directly push the release images to the local registry by using following command:

        $ oc adm release mirror -a ${LOCAL_SECRET_JSON}  \
             --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}-${ARCHITECTURE} \
             --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
             --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}

        This command pulls the release information as a digest, and its output includes the imageContentSources data that you require when you install your cluster.

     2. Record the entire imageContentSources section from the output of the previous command. The information about your mirrors is unique to your mirrored repository, and you must add the imageContentSources section to the install-config.yaml file during installation.

        Note

        The image name gets patched to Quay.io during the mirroring process, and the podman images will show Quay.io in the registry on the bootstrap virtual machine.

4. To create the installation program that is based on the content that you mirrored, extract it and pin it to the release:

   • If your mirror host does not have internet access, run the following command:

     $ oc adm release extract -a ${LOCAL_SECRET_JSON} --icsp-file=<file> --command=openshift-install "${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}" \
     --insecure=true 1

     1

     Optional: If you do not want to configure trust for the target registry, add the --insecure=true flag.

   • If the local container registry is connected to the mirror host, run the following command:

     $ oc adm release extract -a ${LOCAL_SECRET_JSON} --command=openshift-install "${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}-${ARCHITECTURE}"

     Important

     To ensure that you use the correct images for the version of OpenShift Container Platform that you selected, you must extract the installation program from the mirrored content.

     You must perform this step on a machine with an active internet connection.

5. For clusters using installer-provisioned infrastructure, run the following command:

   $ openshift-install

Procedure

1. Use the oc mirror init command to create a template for the image set configuration and save it to a file called imageset-config.yaml:

   $ oc mirror init --registry example.com/mirror/oc-mirror-metadata > imageset-config.yaml 1

   1

   Replace example.com/mirror/oc-mirror-metadata with the location of your registry for the storage backend.

2. Edit the file and adjust the settings as necessary:

   kind: ImageSetConfiguration
   apiVersion: mirror.openshift.io/v1alpha2
   archiveSize: 4                                                      1
   storageConfig:                                                      2
     registry:
       imageURL: example.com/mirror/oc-mirror-metadata                 3
       skipTLS: false
   mirror:
     platform:
       channels:
       - name: stable-4.15                                             4
         type: ocp
       graph: true                                                     5
     operators:
     - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.15  6
       packages:
       - name: serverless-operator                                     7
         channels:
         - name: stable                                                8
     additionalImages:
     - name: registry.redhat.io/ubi9/ubi:latest                        9
     helm: {}

   1

   Add archiveSize to set the maximum size, in GiB, of each file within the image set.

   2

   Set the back-end location to save the image set metadata to. This location can be a registry or local directory. It is required to specify storageConfig values.

   3

   Set the registry URL for the storage backend.

   4

   Set the channel to retrieve the OpenShift Container Platform images from.

   5

   Add graph: true to build and push the graph-data image to the mirror registry. The graph-data image is required to create OpenShift Update Service (OSUS). The graph: true field also generates the UpdateService custom resource manifest. The oc command-line interface (CLI) can use the UpdateService custom resource manifest to create OSUS. For more information, see About the OpenShift Update Service.

   6

   Set the Operator catalog to retrieve the OpenShift Container Platform images from.

   7

   Specify only certain Operator packages to include in the image set. Remove this field to retrieve all packages in the catalog.

   8

   Specify only certain channels of the Operator packages to include in the image set. You must always include the default channel for the Operator package even if you do not use the bundles in that channel. You can find the default channel by running the following command: oc mirror list operators --catalog=<catalog_name> --package=<package_name>.

   9

   Specify any additional images to include in image set.

   Note

   The graph: true field also mirrors the ubi-micro image along with other mirrored images.

   When upgrading OpenShift Container Platform Extended Update Support (EUS) versions, an intermediate version might be required between the current and target versions. For example, if the current version is 4.14 and target version is 4.16, you might need to include a version such as 4.15.8 in the ImageSetConfiguration when using the oc-mirror plugin v1.

   The oc-mirror plugin v1 might not always detect this automatically, so check the Cincinnati graph web page to confirm any required intermediate versions and add them manually to your configuration.

   See Image set configuration parameters for the full list of parameters and Image set configuration examples for various mirroring use cases.

3. Save the updated file.

   This image set configuration file is required by the oc mirror command when mirroring content.

Procedure

1. Log in to the OpenShift CLI as a user with the cluster-admin role.

2. Apply the YAML files from the results directory to the cluster by running the following command:

   $ oc apply -f ./oc-mirror-workspace/results-1639608409/

3. If you mirrored release images, apply the release image signatures to the cluster by running the following command:

   $ oc apply -f ./oc-mirror-workspace/results-1639608409/release-signatures/

   Note

   If you are mirroring Operators instead of clusters, you do not need to run $ oc apply -f ./oc-mirror-workspace/results-1639608409/release-signatures/. Running that command will return an error, as there are no release image signatures to apply.

Verification

1. Verify that the ImageContentSourcePolicy resources were successfully installed by running the following command:

   $ oc get imagecontentsourcepolicy

2. Verify that the CatalogSource resources were successfully installed by running the following command:

   $ oc get catalogsource -n openshift-marketplace

Procedure

1. Run the oc mirror command with the --dry-run flag to perform a dry run:

   $ oc mirror --config=./imageset-config.yaml \1
     docker://registry.example:5000            \2
     --dry-run                                  3

   1

   Pass in the image set configuration file that was created. This procedure assumes that it is named imageset-config.yaml.

   2

   Specify the mirror registry. Nothing is mirrored to this registry as long as you use the --dry-run flag.

   3

   Use the --dry-run flag to generate the dry run artifacts and not an actual image set file.

   Example output

   Checking push permissions for registry.example:5000
   Creating directory: oc-mirror-workspace/src/publish
   Creating directory: oc-mirror-workspace/src/v2
   Creating directory: oc-mirror-workspace/src/charts
   Creating directory: oc-mirror-workspace/src/release-signatures
   No metadata detected, creating new workspace
   wrote mirroring manifests to oc-mirror-workspace/operators.1658342351/manifests-redhat-operator-index
   
   ...
   
   info: Planning completed in 31.48s
   info: Dry run complete
   Writing image mapping to oc-mirror-workspace/mapping.txt

2. Navigate into the workspace directory that was generated:

   $ cd oc-mirror-workspace/

3. Review the mapping.txt file that was generated.

   This file contains a list of all images that would be mirrored.

4. Review the pruning-plan.json file that was generated.

   This file contains a list of all images that would be pruned from the mirror registry when the image set is published.

   Note

   The pruning-plan.json file is only generated if your oc-mirror command points to your mirror registry and there are images to be pruned.

Procedure

1. Create the image set configuration file and adjust the settings as necessary.

   The following example image set configuration mirrors an OCI catalog on disk along with an OpenShift Container Platform release and a UBI image from registry.redhat.io.

   kind: ImageSetConfiguration
   apiVersion: mirror.openshift.io/v1alpha2
   storageConfig:
     local:
       path: /home/user/metadata                                                1
   mirror:
     platform:
       channels:
       - name: stable-4.15                                                      2
         type: ocp
       graph: false
     operators:
     - catalog: oci:///home/user/oc-mirror/my-oci-catalog                       3
       targetCatalog: my-namespace/redhat-operator-index                        4
       packages:
       - name: aws-load-balancer-operator
     - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.15           5
       packages:
       - name: rhacs-operator
     additionalImages:
     - name: registry.redhat.io/ubi9/ubi:latest                                 6

   1

   Set the back-end location to save the image set metadata to. This location can be a registry or local directory. It is required to specify storageConfig values.

   2

   Optionally, include an OpenShift Container Platform release to mirror from registry.redhat.io.

   3

   Specify the absolute path to the location of the OCI catalog on disk. The path must start with oci:// when using the OCI feature.

   4

   Optionally, specify an alternative namespace and name to mirror the catalog as.

   5

   Optionally, specify additional Operator catalogs to pull from a registry.

   6

   Optionally, specify additional images to pull from a registry.

2. Run the oc mirror command to mirror the OCI catalog to a target mirror registry:

   $ oc mirror --config=./imageset-config.yaml \ 1
     docker://registry.example:5000              2

   1

   Pass in the image set configuration file. This procedure assumes that it is named imageset-config.yaml.

   2

   Specify the registry to mirror the content to. The registry must start with docker://. If you specify a top-level namespace for the mirror registry, you must also use this same namespace on subsequent executions.

   Optionally, you can specify other flags to adjust the behavior of the OCI feature:

   --oci-insecure-signature-policy

   Do not push signatures to the target mirror registry.

   --oci-registries-config

   Specify the path to a TOML-formatted registries.conf file. You can use this to mirror from a different registry, such as a pre-production location for testing, without having to change the image set configuration file. This flag only affects local OCI catalogs, not any other mirrored content.

   Example registries.conf file

   [[registry]]
    location = "registry.redhat.io:5000"
    insecure = false
    blocked = false
    mirror-by-digest-only = true
    prefix = ""
    [[registry.mirror]]
       location = "preprod-registry.example.com"
       insecure = false