Chapter 2. Migrating from OpenShift Container Platform 4.1

Procedure

1. In the OpenShift Container Platform web console, click Operators OperatorHub.
2. Use the Filter by keyword field to find the Migration Toolkit for Containers Operator.

3. Select the Migration Toolkit for Containers Operator and click Install.

   Note

   Do not change the subscription approval option to Automatic. The Migration Toolkit for Containers version must be the same on the source and the target clusters.

4. Click Install.

   On the Installed Operators page, the Migration Toolkit for Containers Operator appears in the openshift-migration project with the status Succeeded.

5. Click Migration Toolkit for Containers Operator.
6. Under Provided APIs, locate the Migration Controller tile, and click Create Instance.
7. Click Create.
8. Click Workloads Pods to verify that the MTC pods are running.

Procedure

1. In the OpenShift Container Platform web console, click Catalog OperatorHub.
2. Use the Filter by keyword field to find the Migration Toolkit for Containers Operator.

3. Select the Migration Toolkit for Containers Operator and click Install.

   Note

   Do not change the subscription approval option to Automatic. The Migration Toolkit for Containers version must be the same on the source and the target clusters.

4. Click Install.

   On the Installed Operators page, the Migration Toolkit for Containers Operator appears in the openshift-migration project with the status Succeeded.

5. Click Migration Toolkit for Containers Operator.
6. Under Provided APIs, locate the Migration Controller tile, and click Create Instance.

7. Update the following parameters in the migration-controller custom resource manifest:

   spec:
   ...
     migration_controller: false
     migration_ui: false
   ...
     deprecated_cors_configuration: true 1

   1

   Add the deprecated_cors_configuration parameter and its value.

8. Click Create.
9. Click Workloads Pods to verify that the MTC pods are running.

Procedure

1. On the workstation with unrestricted network access, authenticate with the target mirror registry:

   $ podman login <registry_host_name>

   Also authenticate with registry.redhat.io so that the base image can be pulled during the build:

   $ podman login registry.redhat.io

2. Build a catalog image based on the redhat-operators catalog from Quay.io, tagging and pushing it to your mirror registry:

   $ oc adm catalog build \
       --appregistry-org redhat-operators \1
       --from=registry.redhat.io/openshift4/ose-operator-registry:v4.5 \2
       --filter-by-os="linux/amd64" \3
       --to=<registry_host_name>:<port>/olm/redhat-operators:v1 \4
       [-a ${REG_CREDS}] \5
       [--insecure] \6
       [--auth-token "${AUTH_TOKEN}"] 7

   1

   Organization (namespace) to pull from an App Registry instance.

   2

   Set --from to the ose-operator-registry base image using the tag that matches the target OpenShift Container Platform cluster major and minor version.

   3

   Set --filter-by-os to the operating system and architecture to use for the base image, which must match the target OpenShift Container Platform cluster. Valid values are linux/amd64, linux/ppc64le, and linux/s390x.

   4

   Name your catalog image and include a tag, for example, v1.

   5

   Optional: If required, specify the location of your registry credentials file.

   6

   Optional: If you do not want to configure trust for the target registry, add the --insecure flag.

   7

   Optional: If other application registry catalogs are used that are not public, specify a Quay authentication token.

   Example output

   INFO[0013] loading Bundles                               dir=/var/folders/st/9cskxqs53ll3wdn434vw4cd80000gn/T/300666084/manifests-829192605
   ...
   Pushed sha256:f73d42950021f9240389f99ddc5b0c7f1b533c054ba344654ff1edaf6bf827e3 to example_registry:5000/olm/redhat-operators:v1

   Sometimes invalid manifests are accidentally introduced catalogs provided by Red Hat; when this happens, you might see some errors:

   Example output with errors

   ...
   INFO[0014] directory                                     dir=/var/folders/st/9cskxqs53ll3wdn434vw4cd80000gn/T/300666084/manifests-829192605 file=4.2 load=package
   W1114 19:42:37.876180   34665 builder.go:141] error building database: error loading package into db: fuse-camel-k-operator.v7.5.0 specifies replacement that couldn't be found
   Uploading ... 244.9kB/s

   These errors are usually non-fatal, and if the Operator package mentioned does not contain an Operator you plan to install or a dependency of one, then they can be ignored.

Procedure

1. The oc adm catalog mirror command extracts the contents of your custom Operator catalog image to generate the manifests required for mirroring. You can choose to either:

   • Allow the default behavior of the command to automatically mirror all of the image content to your mirror registry after generating manifests, or
   • Add the --manifests-only flag to only generate the manifests required for mirroring, but do not actually mirror the image content to a registry yet. This can be useful for reviewing what will be mirrored, and it allows you to make any changes to the mapping list if you only require a subset of the content. You can then use that file with the oc image mirror command to mirror the modified list of images in a later step.

   On your workstation with unrestricted network access, run the following command:

   $ oc adm catalog mirror \
       <registry_host_name>:<port>/olm/redhat-operators:v1 \1
       <registry_host_name>:<port> \
       [-a ${REG_CREDS}] \2
       [--insecure] \3
       --filter-by-os='.*' \4
       [--manifests-only] 5

   1

   Specify your Operator catalog image.

   2

   Optional: If required, specify the location of your registry credentials file.

   3

   Optional: If you do not want to configure trust for the target registry, add the --insecure flag.

   4

   This flag is currently required due to a known issue with multiple architecture support.

   5

   Optional: Only generate the manifests required for mirroring and do not actually mirror the image content to a registry.

   Warning

   If the --filter-by-os flag remains unset or set to any value other than .*, the command filters out different architectures, which changes the digest of the manifest list, also known as a multi-arch image. The incorrect digest causes deployments of those images and Operators on disconnected clusters to fail. For more information, see BZ#1890951.

   Example output

   using database path mapping: /:/tmp/190214037
   wrote database to /tmp/190214037
   using database at: /tmp/190214037/bundles.db 1
   ...

   1

   Temporary database generated by the command.

   After running the command, a <image_name>-manifests/ directory is created in the current directory and generates the following files:

   • The imageContentSourcePolicy.yaml file defines an ImageContentSourcePolicy object that can configure nodes to translate between the image references stored in Operator manifests and the mirrored registry.
   • The mapping.txt file contains all of the source images and where to map them in the target registry. This file is compatible with the oc image mirror command and can be used to further customize the mirroring configuration.

2. If you used the --manifests-only flag in the previous step and want to mirror only a subset of the content:

   1. Modify the list of images in your mapping.txt file to your specifications. If you are unsure of the exact names and versions of the subset of images you want to mirror, use the following steps to find them:

      1. Run the sqlite3 tool against the temporary database that was generated by the oc adm catalog mirror command to retrieve a list of images matching a general search query. The output helps inform how you will later edit your mapping.txt file.

         For example, to retrieve a list of images that are similar to the string clusterlogging.4.3:

         $ echo "select * from related_image \
             where operatorbundle_name like 'clusterlogging.4.3%';" \
             | sqlite3 -line /tmp/190214037/bundles.db 1

         1

         Refer to the previous output of the oc adm catalog mirror command to find the path of the database file.

         Example output

         image = registry.redhat.io/openshift4/ose-logging-kibana5@sha256:aa4a8b2a00836d0e28aa6497ad90a3c116f135f382d8211e3c55f34fb36dfe61
         operatorbundle_name = clusterlogging.4.3.33-202008111029.p0
         
         image = registry.redhat.io/openshift4/ose-oauth-proxy@sha256:6b4db07f6e6c962fc96473d86c44532c93b146bbefe311d0c348117bf759c506
         operatorbundle_name = clusterlogging.4.3.33-202008111029.p0
         ...

      2. Use the results from the previous step to edit the mapping.txt file to only include the subset of images you want to mirror.

         For example, you can use the image values from the previous example output to find that the following matching lines exist in your mapping.txt file:

         Matching image mappings in mapping.txt

         registry.redhat.io/openshift4/ose-logging-kibana5@sha256:aa4a8b2a00836d0e28aa6497ad90a3c116f135f382d8211e3c55f34fb36dfe61=<registry_host_name>:<port>/openshift4-ose-logging-kibana5:a767c8f0
         registry.redhat.io/openshift4/ose-oauth-proxy@sha256:6b4db07f6e6c962fc96473d86c44532c93b146bbefe311d0c348117bf759c506=<registry_host_name>:<port>/openshift4-ose-oauth-proxy:3754ea2b

         In this example, if you only want to mirror these images, you would then remove all other entries in the mapping.txt file and leave only the above two lines.

   2. Still on your workstation with unrestricted network access, use your modified mapping.txt file to mirror the images to your registry using the oc image mirror command:

      $ oc image mirror \
          [-a ${REG_CREDS}] \
          --filter-by-os='.*' \
          -f ./redhat-operators-manifests/mapping.txt

      Warning

      If the --filter-by-os flag remains unset or set to any value other than .*, the command filters out different architectures, which changes the digest of the manifest list, also known as a multi-arch image. The incorrect digest causes deployments of those images and Operators on disconnected clusters to fail.

3. Apply the ImageContentSourcePolicy object:

   $ oc apply -f ./redhat-operators-manifests/imageContentSourcePolicy.yaml

4. Create a CatalogSource object that references your catalog image.

   1. Modify the following to your specifications and save it as a catalogsource.yaml file:

      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      metadata:
        name: my-operator-catalog
        namespace: openshift-marketplace
      spec:
        sourceType: grpc
        image: <registry_host_name>:<port>/olm/redhat-operators:v1 1
        displayName: My Operator Catalog
        publisher: grpc

      1

      Specify your custom Operator catalog image.

   2. Use the file to create the CatalogSource object:

      $ oc create -f catalogsource.yaml

5. Verify the following resources are created successfully.

   1. Check the pods:

      $ oc get pods -n openshift-marketplace

      Example output

      NAME                                    READY   STATUS    RESTARTS  AGE
      my-operator-catalog-6njx6               1/1     Running   0         28s
      marketplace-operator-d9f549946-96sgr    1/1     Running   0         26h

   2. Check the catalog source:

      $ oc get catalogsource -n openshift-marketplace

      Example output

      NAME                  DISPLAY               TYPE PUBLISHER  AGE
      my-operator-catalog   My Operator Catalog   grpc            5s

   3. Check the package manifest:

      $ oc get packagemanifest -n openshift-marketplace

      Example output

      NAME    CATALOG              AGE
      etcd    My Operator Catalog  34s

Procedure

1. In the OpenShift Container Platform web console, click Operators OperatorHub.
2. Use the Filter by keyword field to find the Migration Toolkit for Containers Operator.

3. Select the Migration Toolkit for Containers Operator and click Install.

   Note

   Do not change the subscription approval option to Automatic. The Migration Toolkit for Containers version must be the same on the source and the target clusters.

4. Click Install.

   On the Installed Operators page, the Migration Toolkit for Containers Operator appears in the openshift-migration project with the status Succeeded.

5. Click Migration Toolkit for Containers Operator.
6. Under Provided APIs, locate the Migration Controller tile, and click Create Instance.
7. Click Create.
8. Click Workloads Pods to verify that the MTC pods are running.

Procedure

1. Use the Filter by keyword field to find the Migration Toolkit for Containers Operator.

2. Select the Migration Toolkit for Containers Operator and click Install.

   Note

   Do not change the subscription approval option to Automatic. The Migration Toolkit for Containers version must be the same on the source and the target clusters.

3. Click Install.

   On the Installed Operators page, the Migration Toolkit for Containers Operator appears in the openshift-migration project with the status Succeeded.

4. Click Migration Toolkit for Containers Operator.
5. Under Provided APIs, locate the Migration Controller tile, and click Create Instance.
6. Click Create.
7. Click Workloads Pods to verify that the MTC pods are running.

Procedure

1. In the OpenShift Container Platform console, navigate to Operators Installed Operators.

   Operators that have a pending upgrade display an Upgrade available status.

2. Click Migration Toolkit for Containers Operator.
3. Click the Subscription tab. Any upgrades requiring approval are displayed next to Upgrade Status. For example, it might display 1 requires approval.
4. Click 1 requires approval, then click Preview Install Plan.
5. Review the resources that are listed as available for upgrade and click Approve.
6. Navigate back to the Operators Installed Operators page to monitor the progress of the upgrade. When complete, the status changes to Succeeded and Up to date.
7. Click Migration Toolkit for Containers Operator.
8. Under Provided APIs, locate the Migration Controller tile, and click Create Instance.

9. If you are upgrading MTC on a source cluster, update the following parameters in the MigrationController custom resource (CR) manifest:

   spec:
   ...
     migration_controller: false
     migration_ui: false
   ...
     deprecated_cors_configuration: true

   You do not need to update the MigrationController CR manifest on the target cluster.

10. Click Create.
11. Click Workloads Pods to verify that the MTC pods are running.

Procedure

1. Log in to the cluster on which the MigrationController pod is running.

2. Get the MigPlan CR manifest:

   $ oc get migplan <migplan> -o yaml -n openshift-migration

3. Update the following parameter values and save the file as migplan.yaml:

   ...
   spec:
     indirectImageMigration: true
     indirectVolumeMigration: true

4. Replace the MigPlan CR manifest to apply the changes:

   $ oc replace -f migplan.yaml -n openshift-migration

5. Get the updated MigPlan CR manifest to verify the changes:

   $ oc get migplan <migplan> -o yaml -n openshift-migration

Procedure

1. In the OpenShift Container Platform web console, click Operators OperatorHub.
2. Use Filter by keyword (in this case, OCS) to find the OpenShift Container Storage Operator.
3. Select the OpenShift Container Storage Operator and click Install.
4. Select an Update Channel, Installation Mode, and Approval Strategy.

5. Click Install.

   On the Installed Operators page, the OpenShift Container Storage Operator appears in the openshift-storage project with the status Succeeded.

Procedure

1. Log in to the OpenShift Container Platform cluster:

   $ oc login

2. Create the NooBaa CR configuration file, noobaa.yml, with the following content:

   apiVersion: noobaa.io/v1alpha1
   kind: NooBaa
   metadata:
     name: noobaa
     namespace: openshift-storage
   spec:
    dbResources:
      requests:
        cpu: 0.5 1
        memory: 1Gi
    coreResources:
      requests:
        cpu: 0.5 2
        memory: 1Gi

   1 2

   For a very small cluster, you can change the cpu value to 0.1.

3. Create the NooBaa object:

   $ oc create -f noobaa.yml

4. Create the BackingStore CR configuration file, bs.yml, with the following content:

   apiVersion: noobaa.io/v1alpha1
   kind: BackingStore
   metadata:
     finalizers:
     - noobaa.io/finalizer
     labels:
       app: noobaa
     name: mcg-pv-pool-bs
     namespace: openshift-storage
   spec:
     pvPool:
       numVolumes: 3 1
       resources:
         requests:
           storage: 50Gi 2
       storageClass: gp2 3
     type: pv-pool

   1

   Specify the number of volumes in the persistent volume pool.

   2

   Specify the size of the volumes.

   3

   Specify the storage class.

5. Create the BackingStore object:

   $ oc create -f bs.yml

6. Create the BucketClass CR configuration file, bc.yml, with the following content:

   apiVersion: noobaa.io/v1alpha1
   kind: BucketClass
   metadata:
     labels:
       app: noobaa
     name: mcg-pv-pool-bc
     namespace: openshift-storage
   spec:
     placementPolicy:
       tiers:
       - backingStores:
         - mcg-pv-pool-bs
         placement: Spread

7. Create the BucketClass object:

   $ oc create -f bc.yml

8. Create the ObjectBucketClaim CR configuration file, obc.yml, with the following content:

   apiVersion: objectbucket.io/v1alpha1
   kind: ObjectBucketClaim
   metadata:
     name: migstorage
     namespace: openshift-storage
   spec:
     bucketName: migstorage 1
     storageClassName: openshift-storage.noobaa.io
     additionalConfig:
       bucketclass: mcg-pv-pool-bc

   1

   Record the bucket name for adding the replication repository to the MTC web console.

9. Create the ObjectBucketClaim object:

   $ oc create -f obc.yml

10. Watch the resource creation process to verify that the ObjectBucketClaim status is Bound:

    $ watch -n 30 'oc get -n openshift-storage objectbucketclaim migstorage -o yaml'

    This process can take five to ten minutes.

11. Obtain and record the following values, which are required when you add the replication repository to the MTC web console:

    • S3 endpoint:

      $ oc get route -n openshift-storage s3

    • S3 provider access key:

      $ oc get secret -n openshift-storage migstorage -o go-template='{{ .data.AWS_ACCESS_KEY_ID }}' | base64 --decode

    • S3 provider secret access key:

      $ oc get secret -n openshift-storage migstorage -o go-template='{{ .data.AWS_SECRET_ACCESS_KEY }}' | base64 --decode

Procedure

1. Log in to the cluster on which the MigrationController pod runs.

2. Get the MigrationController CR manifest:

   $ oc get migrationcontroller <migration_controller> -n openshift-migration

3. Add the stunnel_tcp_proxy parameter:

   apiVersion: migration.openshift.io/v1alpha1
   kind: MigrationController
   metadata:
     name: migration-controller
     namespace: openshift-migration
   ...
   spec:
     stunnel_tcp_proxy: <stunnel_proxy> 1

   1

   Specify the Stunnel proxy: http://<user_name>:<password>@<ip_address>:<port>.

4. Save the manifest as migration-controller.yaml.

5. Apply the updated manifest:

   $ oc replace -f migration-controller.yaml -n openshift-migration

Procedure

1. Log in to the OpenShift Container Platform cluster on which you have installed MTC.

2. Obtain the MTC web console URL by entering the following command:

   $ oc get -n openshift-migration route/migration -o go-template='https://{{ .spec.host }}'

   The output resembles the following: https://migration-openshift-migration.apps.cluster.openshift.com.

3. Launch a browser and navigate to the MTC web console.

   Note

   If you try to access the MTC web console immediately after installing the Migration Toolkit for Containers Operator, the console might not load because the Operator is still configuring the cluster. Wait a few minutes and retry.

4. If you are using self-signed CA certificates, you will be prompted to accept the CA certificate of the source cluster API server. The web page guides you through the process of accepting the remaining certificates.
5. Log in with your OpenShift Container Platform username and password.

Procedure

1. Log in to the cluster.

2. Obtain the migration-controller service account token:

   $ oc sa get-token migration-controller -n openshift-migration

   Example output

   eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtaWciLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoibWlnLXRva2VuLWs4dDJyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6Im1pZyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImE1YjFiYWMwLWMxYmYtMTFlOS05Y2NiLTAyOWRmODYwYjMwOCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDptaWc6bWlnIn0.xqeeAINK7UXpdRqAtOj70qhBJPeMwmgLomV9iFxr5RoqUgKchZRG2J2rkqmPm6vr7K-cm7ibD1IBpdQJCcVDuoHYsFgV4mp9vgOfn9osSDp2TGikwNz4Az95e81xnjVUmzh-NjDsEpw71DH92iHV_xt2sTwtzftS49LpPW2LjrV0evtNBP_t_RfskdArt5VSv25eORl7zScqfe1CiMkcVbf2UqACQjo3LbkpfN26HAioO2oH0ECPiRzT0Xyh-KwFutJLS9Xgghyw-LD9kPKcE_xbbJ9Y4Rqajh7WdPYuB0Jd9DPVrslmzK-F6cgHHYoZEv0SvLQi-PO0rpDrcjOEQQ

3. In the MTC web console, click Clusters.
4. Click Add cluster.

5. Fill in the following fields:

   • Cluster name: The cluster name can contain lower-case letters (a-z) and numbers (0-9). It must not contain spaces or international characters.
   • URL: Specify the API server URL, for example, https://<www.example.com>:8443.
   • Service account token: Paste the migration-controller service account token.

   • Exposed route host to image registry: If you are using direct image migration, specify the exposed route to the image registry of the source cluster, for example, www.example.apps.cluster.com.

     You can specify a port. The default port is 5000.

   • Azure cluster: You must select this option if you use Azure snapshots to copy your data.
   • Azure resource group: This field is displayed if Azure cluster is selected. Specify the Azure resource group.
   • Require SSL verification: Optional: Select this option to verify SSL connections to the cluster.
   • CA bundle file: This field is displayed if Require SSL verification is selected. If you created a custom CA certificate bundle file for self-signed certificates, click Browse, select the CA bundle file, and upload it.

6. Click Add cluster.

   The cluster appears in the Clusters list.

Procedure

1. In the MTC web console, click Replication repositories.
2. Click Add repository.

3. Select a Storage provider type and fill in the following fields:

   • AWS for AWS S3, MCG, and generic S3 providers:

     • Replication repository name: Specify the replication repository name in the MTC web console.
     • S3 bucket name: Specify the name of the S3 bucket you created.
     • S3 bucket region: Specify the S3 bucket region. Required for AWS S3. Optional for other S3 providers.
     • S3 endpoint: Specify the URL of the S3 service, not the bucket, for example, https://<s3-storage.apps.cluster.com>. Required for a generic S3 provider. You must use the https:// prefix.
     • S3 provider access key: Specify the <AWS_SECRET_ACCESS_KEY> for AWS or the S3 provider access key for MCG.
     • S3 provider secret access key: Specify the <AWS_ACCESS_KEY_ID> for AWS or the S3 provider secret access key for MCG.
     • Require SSL verification: Clear this check box if you are using a generic S3 provider.
     • If you use a custom CA bundle, click Browse and browse to the Base64-encoded CA bundle file.

   • GCP:

     • Replication repository name: Specify the replication repository name in the MTC web console.
     • GCP bucket name: Specify the name of the GCP bucket.
     • GCP credential JSON blob: Specify the string in the credentials-velero file.

   • Azure:

     • Replication repository name: Specify the replication repository name in the MTC web console.
     • Azure resource group: Specify the resource group of the Azure Blob storage.
     • Azure storage account name: Specify the Azure Blob storage account name.
     • Azure credentials - INI file contents: Specify the string in the credentials-velero file.
4. Click Add repository and wait for connection validation.

5. Click Close.

   The new repository appears in the Replication repositories list.

Procedure

1. In the MTC web console, click Migration plans.
2. Click Add migration plan.

3. Enter the Plan name and click Next.

   The migration plan name must not exceed 253 lower-case alphanumeric characters (a-z, 0-9) and must not contain spaces or underscores (_).

4. Select a Source cluster.
5. Select a Target cluster.
6. Select a Replication repository.
7. Select the projects to be migrated and click Next.
8. Select a Source cluster, a Target cluster, and a Repository, and click Next.
9. On the Namespaces page, select the projects to be migrated and click Next.

10. On the Persistent volumes page, click a Migration type for each PV:

    • The Copy option copies the data from the PV of a source cluster to the replication repository and then restores the data on a newly created PV, with similar characteristics, in the target cluster.
    • The Move option unmounts a remote volume, for example, NFS, from the source cluster, creates a PV resource on the target cluster pointing to the remote volume, and then mounts the remote volume on the target cluster. Applications running on the target cluster use the same remote volume that the source cluster was using.
11. Click Next.

12. On the Copy options page, select a Copy method for each PV:

    • Snapshot copy backs up and restores data using the cloud provider’s snapshot functionality. It is significantly faster than Filesystem copy.

    • Filesystem copy backs up the files on the source cluster and restores them on the target cluster.

      The file system copy method is required for direct volume migration.

13. You can select Verify copy to verify data migrated with Filesystem copy. Data is verified by generating a checksum for each source file and checking the checksum after restoration. Data verification significantly reduces performance.

14. Select a Target storage class.

    If you selected Filesystem copy, you can change the target storage class.

15. Click Next.

16. On the Migration options page, the Direct image migration option is selected if you specified an exposed image registry route for the source cluster. The Direct PV migration option is selected if you are migrating data with Filesystem copy.

    The direct migration options copy images and files directly from the source cluster to the target cluster. This option is much faster than copying images and files from the source cluster to the replication repository and then from the replication repository to the target cluster.

17. Click Next.

18. Optional: On the Hooks page, click Add Hook to add a hook to the migration plan.

    A hook runs custom code. You can add up to four hooks to a single migration plan. Each hook runs during a different migration step.

    1. Enter the name of the hook to display in the web console.
    2. If the hook is an Ansible playbook, select Ansible playbook and click Browse to upload the playbook or paste the contents of the playbook in the field.
    3. Optional: Specify an Ansible runtime image if you are not using the default hook image.

    4. If the hook is not an Ansible playbook, select Custom container image and specify the image name and path.

       A custom container image can include Ansible playbooks.

    5. Select Source cluster or Target cluster.
    6. Enter the Service account name and the Service account namespace.

    7. Select the migration step for the hook:

       • preBackup: Before the application workload is backed up on the source cluster
       • postBackup: After the application workload is backed up on the source cluster
       • preRestore: Before the application workload is restored on the target cluster
       • postRestore: After the application workload is restored on the target cluster
    8. Click Add.

19. Click Finish.

    The migration plan is displayed in the Migration plans list.

Procedure

1. Log in to the source cluster.

2. Delete old images:

   $ oc adm prune images

3. Log in to the MTC web console and click Migration plans.

4. Click the Options menu next to a migration plan and select Stage to copy data from the source cluster to the target cluster without stopping the application.

   You can run Stage multiple times to reduce the actual migration time.

5. When you are ready to migrate the application workload, the Options menu beside a migration plan and select Migrate.
6. Optional: In the Migrate window, you can select Do not stop applications on the source cluster during migration.
7. Click Migrate.

8. When the migration is complete, verify that the application migrated successfully in the OpenShift Container Platform web console:

   1. Click Home Projects.
   2. Click the migrated project to view its status.
   3. In the Routes section, click Location to verify that the application is functioning, if applicable.
   4. Click Workloads Pods to verify that the pods are running in the migrated namespace.
   5. Click Storage Persistent volumes to verify that the migrated persistent volume is correctly provisioned.

Procedure

1. Create a MigCluster CR manifest for the host cluster called host-cluster.yaml:

   apiVersion: migration.openshift.io/v1alpha1
   kind: MigCluster
   metadata:
     name: host
     namespace: openshift-migration
   spec:
     isHostCluster: true

2. Create a MigCluster CR for the host cluster:

   $ oc create -f host-cluster.yaml -n openshift-migration

3. Create a Secret CR manifest for each remote cluster called cluster-secret.yaml:

   apiVersion: v1
   kind: Secret
   metadata:
     name: <cluster_secret>
     namespace: openshift-config
   type: Opaque
   data:
     saToken: <sa_token> 1

   1

   Specify the base64-encoded migration-controller service account (SA) token of the remote cluster.

   You can obtain the SA token by running the following command:

   $ oc sa get-token migration-controller -n openshift-migration | base64 -w 0

4. Create a Secret CR for each remote cluster:

   $ oc create -f cluster-secret.yaml

5. Create a MigCluster CR manifest for each remote cluster called remote-cluster.yaml:

   apiVersion: migration.openshift.io/v1alpha1
   kind: MigCluster
   metadata:
     name: <remote_cluster>
     namespace: openshift-migration
   spec:
     exposedRegistryPath: <exposed_registry_route> 1
     insecure: false 2
     isHostCluster: false
     serviceAccountSecretRef:
       name: <remote_cluster_secret> 3
       namespace: openshift-config
     url: <remote_cluster_url> 4

   1

   Optional: Specify the exposed registry route, for example, docker-registry-default.apps.example.com if you are using direct image migration.

   2

   SSL verification is enabled if false. CA certificates are not required or checked if true.

   3

   Specify the Secret CR of the remote cluster.

   4

   Specify the URL of the remote cluster.

6. Create a MigCluster CR for each remote cluster:

   $ oc create -f remote-cluster.yaml -n openshift-migration

7. Verify that all clusters are in a Ready state:

   $ oc describe cluster <cluster_name>

8. Create a Secret CR manifest for the replication repository called storage-secret.yaml:

   apiVersion: v1
   kind: Secret
   metadata:
     namespace: openshift-config
     name: <migstorage_creds>
   type: Opaque
   data:
     aws-access-key-id: <key_id_base64> 1
     aws-secret-access-key: <secret_key_base64> 2

   1

   Specify the key ID in base64 format.

   2

   Specify the secret key in base64 format.

   AWS credentials are base64-encoded by default. If you are using another storage provider, you must encode your credentials by running the following command with each key:

   $ echo -n "<key>" | base64 -w 0 1

   1

   Specify the key ID or the secret key. Both keys must be base64-encoded.

9. Create the Secret CR for the replication repository:

   $ oc create -f storage-secret.yaml

10. Create a MigStorage CR manifest for the replication repository called migstorage.yaml:

    apiVersion: migration.openshift.io/v1alpha1
    kind: MigStorage
    metadata:
      name: <storage_name>
      namespace: openshift-migration
    spec:
      backupStorageConfig:
        awsBucketName: <bucket_name> 1
        credsSecretRef:
          name: <storage_secret_ref> 2
          namespace: openshift-config
      backupStorageProvider: <storage_provider_name> 3
      volumeSnapshotConfig:
        credsSecretRef:
          name: <storage_secret_ref> 4
          namespace: openshift-config
      volumeSnapshotProvider: <storage_provider_name> 5

    1

    Specify the bucket name.

    2

    Specify the Secrets CR of the object storage. You must ensure that the credentials stored in the Secrets CR of the object storage are correct.

    3

    Specify the storage provider.

    4

    Optional: If you are copying data by using snapshots, specify the Secrets CR of the object storage. You must ensure that the credentials stored in the Secrets CR of the object storage are correct.

    5

    Optional: If you are copying data by using snapshots, specify the storage provider.

11. Create the MigStorage CR:

    $ oc create -f migstorage.yaml -n openshift-migration

12. Verify that the MigStorage CR is in a Ready state:

    $ oc describe migstorage <migstorage_name>

13. Create a MigPlan CR manifest called migplan.yaml:

    apiVersion: migration.openshift.io/v1alpha1
    kind: MigPlan
    metadata:
      name: <migration_plan>
      namespace: openshift-migration
    spec:
      destMigClusterRef:
        name: host
        namespace: openshift-migration
      indirectImageMigration: true 1
      indirectVolumeMigration: true 2
      migStorageRef:
        name: <migstorage_ref> 3
        namespace: openshift-migration
      namespaces:
        - <application_namespace> 4
      srcMigClusterRef:
        name: <remote_cluster_ref> 5
        namespace: openshift-migration

    1

    Direct image migration is enabled if false.

    2

    Direct volume migration is enabled if false.

    3

    Specify the name of the MigStorage CR instance.

    4

    Specify one or more namespaces to be migrated.

    5

    Specify the name of the source cluster MigCluster instance.

14. Create the MigPlan CR:

    $ oc create -f migplan.yaml -n openshift-migration

15. View the MigPlan instance to verify that it is in a Ready state:

    $ oc describe migplan <migplan_name> -n openshift-migration

16. Create a MigMigration CR manifest called migmigration.yaml:

    apiVersion: migration.openshift.io/v1alpha1
    kind: MigMigration
    metadata:
      name: <migmigration_name>
      namespace: openshift-migration
    spec:
      migPlanRef:
        name: <migplan_name> 1
        namespace: openshift-migration
      quiescePods: true 2
      stage: false 3
      rollback: false 4

    1

    Specify the MigPlan CR name.

    2

    The pods on the source cluster are stopped before migration if true.

    3

    A stage migration, which copies most of the data without stopping the application, is performed if true.

    4

    A completed migration is rolled back if true.

17. Create the MigMigration CR to start the migration defined in the MigPlan CR:

    $ oc create -f migmigration.yaml -n openshift-migration

18. Verify the progress of the migration by watching the MigMigration CR:

    $ oc watch migmigration <migmigration_name> -n openshift-migration

    The output resembles the following:

    Example output

    Name:         c8b034c0-6567-11eb-9a4f-0bc004db0fbc
    Namespace:    openshift-migration
    Labels:       migration.openshift.io/migplan-name=django
    Annotations:  openshift.io/touch: e99f9083-6567-11eb-8420-0a580a81020c
    API Version:  migration.openshift.io/v1alpha1
    Kind:         MigMigration
    ...
    Spec:
      Mig Plan Ref:
        Name:       my_application
        Namespace:  openshift-migration
      Stage:        false
    Status:
      Conditions:
        Category:              Advisory
        Last Transition Time:  2021-02-02T15:04:09Z
        Message:               Step: 19/47
        Reason:                InitialBackupCreated
        Status:                True
        Type:                  Running
        Category:              Required
        Last Transition Time:  2021-02-02T15:03:19Z
        Message:               The migration is ready.
        Status:                True
        Type:                  Ready
        Category:              Required
        Durable:               true
        Last Transition Time:  2021-02-02T15:04:05Z
        Message:               The migration registries are healthy.
        Status:                True
        Type:                  RegistriesHealthy
      Itinerary:               Final
      Observed Digest:         7fae9d21f15979c71ddc7dd075cb97061895caac5b936d92fae967019ab616d5
      Phase:                   InitialBackupCreated
      Pipeline:
        Completed:  2021-02-02T15:04:07Z
        Message:    Completed
        Name:       Prepare
        Started:    2021-02-02T15:03:18Z
        Message:    Waiting for initial Velero backup to complete.
        Name:       Backup
        Phase:      InitialBackupCreated
        Progress:
          Backup openshift-migration/c8b034c0-6567-11eb-9a4f-0bc004db0fbc-wpc44: 0 out of estimated total of 0 objects backed up (5s)
        Started:        2021-02-02T15:04:07Z
        Message:        Not started
        Name:           StageBackup
        Message:        Not started
        Name:           StageRestore
        Message:        Not started
        Name:           DirectImage
        Message:        Not started
        Name:           DirectVolume
        Message:        Not started
        Name:           Restore
        Message:        Not started
        Name:           Cleanup
      Start Timestamp:  2021-02-02T15:03:18Z
    Events:
      Type    Reason   Age                 From                     Message
      ----    ------   ----                ----                     -------
      Normal  Running  57s                 migmigration_controller  Step: 2/47
      Normal  Running  57s                 migmigration_controller  Step: 3/47
      Normal  Running  57s (x3 over 57s)   migmigration_controller  Step: 4/47
      Normal  Running  54s                 migmigration_controller  Step: 5/47
      Normal  Running  54s                 migmigration_controller  Step: 6/47
      Normal  Running  52s (x2 over 53s)   migmigration_controller  Step: 7/47
      Normal  Running  51s (x2 over 51s)   migmigration_controller  Step: 8/47
      Normal  Ready    50s (x12 over 57s)  migmigration_controller  The migration is ready.
      Normal  Running  50s                 migmigration_controller  Step: 9/47
      Normal  Running  50s                 migmigration_controller  Step: 10/47

Procedure

1. Edit the MigrationController custom resource (CR) manifest:

   $ oc edit migrationcontroller -n openshift-migration

2. Update the following parameters:

   ...
   mig_controller_limits_cpu: "1" 1
   mig_controller_limits_memory: "10Gi" 2
   ...
   mig_controller_requests_cpu: "100m" 3
   mig_controller_requests_memory: "350Mi" 4
   ...
   mig_pv_limit: 100 5
   mig_pod_limit: 100 6
   mig_namespace_limit: 10 7
   ...

   1

   Specifies the number of CPUs available to the MigrationController CR.

   2

   Specifies the amount of memory available to the MigrationController CR.

   3

   Specifies the number of CPU units available for MigrationController CR requests. 100m represents 0.1 CPU units (100 * 1e-3).

   4

   Specifies the amount of memory available for MigrationController CR requests.

   5

   Specifies the number of persistent volumes that can be migrated.

   6

   Specifies the number of pods that can be migrated.

   7

   Specifies the number of namespaces that can be migrated.

3. Create a migration plan that uses the updated parameters to verify the changes.

   If your migration plan exceeds the MigrationController CR limits, the MTC console displays a warning message when you save the migration plan.

Procedure

1. Edit the MigrationController custom resource manifest:

   $ oc edit migrationcontroller <migration_controller> -n openshift-migration

2. Update the spec section by adding a parameter to exclude specific resources or by adding a resource to the excluded_resources parameter if it does not have its own exclusion parameter:

   apiVersion: migration.openshift.io/v1alpha1
   kind: MigrationController
   metadata:
     name: migration-controller
     namespace: openshift-migration
   spec:
     disable_image_migration: true 1
     disable_pv_migration: true 2
     ...
     excluded_resources: 3
     - imagetags
     - templateinstances
     - clusterserviceversions
     - packagemanifests
     - subscriptions
     - servicebrokers
     - servicebindings
     - serviceclasses
     - serviceinstances
     - serviceplans
     - operatorgroups
     - events

   1

   Add disable_image_migration: true to exclude image streams from the migration. Do not edit the excluded_resources parameter. imagestreams is added to excluded_resources when the MigrationController pod restarts.

   2

   Add disable_pv_migration: true to exclude PVs from the migration plan. Do not edit the excluded_resources parameter. persistentvolumes and persistentvolumeclaims are added to excluded_resources when the MigrationController pod restarts. Disabling PV migration also disables PV discovery when you create the migration plan.

   3

   You can add OpenShift Container Platform resources to the excluded_resources list. Do not delete the default excluded resources. These resources are problematic to migrate and must be excluded.

3. Wait two minutes for the MigrationController pod to restart so that the changes are applied.

4. Verify that the resource is excluded:

   $ oc get deployment -n openshift-migration migration-controller -o yaml | grep EXCLUDED_RESOURCES -A1

   The output contains the excluded resources:

   Example output

       - name: EXCLUDED_RESOURCES
         value:
         imagetags,templateinstances,clusterserviceversions,packagemanifests,subscriptions,servicebrokers,servicebindings,serviceclasses,serviceinstances,serviceplans,imagestreams,persistentvolumes,persistentvolumeclaims

Procedure

1. Navigate to the MTC console and inspect the elements with the browser web inspector.

2. Check the MigrationUI pod log:

   $ oc logs <MigrationUI_Pod> -n openshift-migration

Procedure

1. In the OpenShift Container Platform web console, navigate to Operators Installed Operators.
2. Click Migration Toolkit for Containers Operator.
3. In the MigrationController tab, click migration-controller.

4. In the YAML tab, update the following parameter value:

   spec:
     restic_timeout: 1h 1

   1

   Valid units are h (hours), m (minutes), and s (seconds), for example, 3h30m15s.

5. Click Save.

Procedure

1. Log in to the target cluster.

2. View the Restore CR:

   $ oc describe <registry-example-migration-rvwcm> -n openshift-migration

   The output identifies the persistent volume with PodVolumeRestore errors.

   Example output

   status:
     phase: Completed
     podVolumeRestoreErrors:
     - kind: PodVolumeRestore
       name: <registry-example-migration-rvwcm-98t49>
       namespace: openshift-migration
     podVolumeRestoreResticErrors:
     - kind: PodVolumeRestore
       name: <registry-example-migration-rvwcm-98t49>
       namespace: openshift-migration

3. View the PodVolumeRestore CR:

   $ oc describe <migration-example-rvwcm-98t49>

   The output identifies the Restic pod that logged the errors.

   Example output

     completionTimestamp: 2020-05-01T20:49:12Z
     errors: 1
     resticErrors: 1
     ...
     resticPod: <restic-nr2v5>

4. View the Restic pod log to locate the errors:

   $ oc logs -f <restic-nr2v5>

Procedure

1. Check the status of a MigMigration CR:

   $ oc get migmigration <migmigration> -o yaml

   Example output

   status:
     conditions:
     - category: Warn
       durable: true
       lastTransitionTime: "2021-01-26T20:48:40Z"
       message: 'Final Restore openshift-migration/ccc7c2d0-6017-11eb-afab-85d0007f5a19-x4lbf: partially failed on destination cluster'
       status: "True"
       type: VeleroFinalRestorePartiallyFailed
     - category: Advisory
       durable: true
       lastTransitionTime: "2021-01-26T20:48:42Z"
       message: The migration has completed with warnings, please look at `Warn` conditions.
       reason: Completed
       status: "True"
       type: SucceededWithWarnings

2. Check the status of the Restore CR by using the Velero describe command:

   $ oc exec $(oc get pods -n openshift-migration -o name | grep velero) -n openshift-migration -- ./velero restore describe <restore>

   Example output

   Phase:  PartiallyFailed (run 'velero restore logs ccc7c2d0-6017-11eb-afab-85d0007f5a19-x4lbf' for more information)
   
   Errors:
     Velero:     <none>
     Cluster:    <none>
     Namespaces:
       migration-example:  error restoring example.com/migration-example/migration-example: the server could not find the requested resource

3. Check the Restore CR logs by using the Velero logs command:

   $ oc exec $(oc get pods -n openshift-migration -o name | grep velero) -n openshift-migration -- ./velero restore logs <restore>

   Example output

   time="2021-01-26T20:48:37Z" level=info msg="Attempting to restore migration-example: migration-example" logSource="pkg/restore/restore.go:1107" restore=openshift-migration/ccc7c2d0-6017-11eb-afab-85d0007f5a19-x4lbf
   time="2021-01-26T20:48:37Z" level=info msg="error restoring migration-example: the server could not find the requested resource" logSource="pkg/restore/restore.go:1170" restore=openshift-migration/ccc7c2d0-6017-11eb-afab-85d0007f5a19-x4lbf

   The Restore CR log error message, the server could not find the requested resource, indicates the cause of the partially failed migration.

Procedure

1. In the MTC web console, click Migration plans.
2. Click the Options menu beside a migration plan and select Rollback.

3. Click Rollback and wait for rollback to complete.

   In the migration plan details, Rollback succeeded is displayed.

4. Verify that rollback was successful in the OpenShift Container Platform web console of the source cluster:

   1. Click Home Projects.
   2. Click the migrated project to view its status.
   3. In the Routes section, click Location to verify that the application is functioning, if applicable.
   4. Click Workloads Pods to verify that the pods are running in the migrated namespace.
   5. Click Storage Persistent volumes to verify that the migrated persistent volume is correctly provisioned.