15.2. 为单节点 OpenShift 集群准备基于镜像的升级

流程

1. 创建 storage.bu 文件：

   variant: fcos
   version: 1.3.0
   storage:
     disks:
     - device: /dev/disk/by-path/pci-<root_disk> 

   1

   
       wipe_table: false
       partitions:
       - label: var-lib-containers
         start_mib: <start_of_partition> 

   2

   
         size_mib: <partition_size> 

   3

   
     filesystems:
       - path: /var/lib/containers
         device: /dev/disk/by-partlabel/var-lib-containers
         format: xfs
         wipe_filesystem: true
         with_mount_unit: true
         mount_options:
           - defaults
           - prjquota

   Copy to Clipboard Toggle word wrap

   1

   指定根磁盘。

   2

   以 MiB 为单位指定分区的起始位置。如果该值太小，则安装将失败。

   3

   为 500 GB 分区指定最小大小，以确保为 precached 镜像有足够的磁盘空间。如果值太小，则安装后部署将失败。

2. 运行以下命令，将 storage.bu 转换为 Ignition 文件：

   $ butane storage.bu

   Copy to Clipboard Toggle word wrap

   输出示例

   {"ignition":{"version":"3.2.0"},"storage":{"disks":[{"device":"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0","partitions":[{"label":"var-lib-containers","sizeMiB":0,"startMiB":250000}],"wipeTable":false}],"filesystems":[{"device":"/dev/disk/by-partlabel/var-lib-containers","format":"xfs","mountOptions":["defaults","prjquota"],"path":"/var/lib/containers","wipeFilesystem":true}]},"systemd":{"units":[{"contents":"# Generated by Butane\n[Unit]\nRequires=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\nAfter=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\n\n[Mount]\nWhere=/var/lib/containers\nWhat=/dev/disk/by-partlabel/var-lib-containers\nType=xfs\nOptions=defaults,prjquota\n\n[Install]\nRequiredBy=local-fs.target","enabled":true,"name":"var-lib-containers.mount"}]}}

   Copy to Clipboard Toggle word wrap

3. 将输出复制到 SiteConfig CR 中的 .spec.clusters.nodes.ignitionConfigOverride 字段中：

   [...]
   spec:
     clusters:
       - nodes:
           - hostName: <name>
             ignitionConfigOverride: '{"ignition":{"version":"3.2.0"},"storage":{"disks":[{"device":"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0","partitions":[{"label":"var-lib-containers","sizeMiB":0,"startMiB":250000}],"wipeTable":false}],"filesystems":[{"device":"/dev/disk/by-partlabel/var-lib-containers","format":"xfs","mountOptions":["defaults","prjquota"],"path":"/var/lib/containers","wipeFilesystem":true}]},"systemd":{"units":[{"contents":"# Generated by Butane\n[Unit]\nRequires=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\nAfter=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\n\n[Mount]\nWhere=/var/lib/containers\nWhat=/dev/disk/by-partlabel/var-lib-containers\nType=xfs\nOptions=defaults,prjquota\n\n[Install]\nRequiredBy=local-fs.target","enabled":true,"name":"var-lib-containers.mount"}]}}'
   [...]

   Copy to Clipboard Toggle word wrap

验证

1. 在安装过程中，运行以下命令来在 hub 集群上验证 BareMetalHost 对象显示注解：

   $ oc get bmh -n my-sno-ns my-sno -ojson | jq '.metadata.annotations["bmac.agent-install.openshift.io/ignition-config-overrides"]'

   Copy to Clipboard Toggle word wrap

   输出示例

   "{\"ignition\":{\"version\":\"3.2.0\"},\"storage\":{\"disks\":[{\"device\":\"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0\",\"partitions\":[{\"label\":\"var-lib-containers\",\"sizeMiB\":0,\"startMiB\":250000}],\"wipeTable\":false}],\"filesystems\":[{\"device\":\"/dev/disk/by-partlabel/var-lib-containers\",\"format\":\"xfs\",\"mountOptions\":[\"defaults\",\"prjquota\"],\"path\":\"/var/lib/containers\",\"wipeFilesystem\":true}]},\"systemd\":{\"units\":[{\"contents\":\"# Generated by Butane\\n[Unit]\\nRequires=systemd-fsck@dev-disk-by\\\\x2dpartlabel-var\\\\x2dlib\\\\x2dcontainers.service\\nAfter=systemd-fsck@dev-disk-by\\\\x2dpartlabel-var\\\\x2dlib\\\\x2dcontainers.service\\n\\n[Mount]\\nWhere=/var/lib/containers\\nWhat=/dev/disk/by-partlabel/var-lib-containers\\nType=xfs\\nOptions=defaults,prjquota\\n\\n[Install]\\nRequiredBy=local-fs.target\",\"enabled\":true,\"name\":\"var-lib-containers.mount\"}]}}"

   Copy to Clipboard Toggle word wrap

2. 安装后，运行以下命令来检查单节点 OpenShift 磁盘状态：

   # lsblk

   Copy to Clipboard Toggle word wrap

   输出示例

   NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
   sda      8:0    0 446.6G  0 disk
   ├─sda1   8:1    0     1M  0 part
   ├─sda2   8:2    0   127M  0 part
   ├─sda3   8:3    0   384M  0 part /boot
   ├─sda4   8:4    0 243.6G  0 part /var
   │                                /sysroot/ostree/deploy/rhcos/var
   │                                /usr
   │                                /etc
   │                                /
   │                                /sysroot
   └─sda5   8:5    0 202.5G  0 part /var/lib/containers

   Copy to Clipboard Toggle word wrap

   # df -h

   Copy to Clipboard Toggle word wrap

   输出示例

   Filesystem      Size  Used Avail Use% Mounted on
   devtmpfs        4.0M     0  4.0M   0% /dev
   tmpfs           126G   84K  126G   1% /dev/shm
   tmpfs            51G   93M   51G   1% /run
   /dev/sda4       244G  5.2G  239G   3% /sysroot
   tmpfs           126G  4.0K  126G   1% /tmp
   /dev/sda5       203G  119G   85G  59% /var/lib/containers
   /dev/sda3       350M  110M  218M  34% /boot
   tmpfs            26G     0   26G   0% /run/user/1000

   Copy to Clipboard Toggle word wrap

流程

1. 为 Lifecycle Agent 创建 Namespace 对象 YAML 文件：

   apiVersion: v1
   kind: Namespace
   metadata:
     name: openshift-lifecycle-agent
     annotations:
       workload.openshift.io/allowed: management

   Copy to Clipboard Toggle word wrap

   1. 运行以下命令来创建 Namespace CR：

      $ oc create -f <namespace_filename>.yaml

      Copy to Clipboard Toggle word wrap

2. 为 Lifecycle Agent 创建 OperatorGroup 对象 YAML 文件：

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: openshift-lifecycle-agent
     namespace: openshift-lifecycle-agent
   spec:
     targetNamespaces:
     - openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   1. 运行以下命令来创建 OperatorGroup CR:

      $ oc create -f <operatorgroup_filename>.yaml

      Copy to Clipboard Toggle word wrap

3. 为生命周期代理创建一个 Subscription CR：

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: openshift-lifecycle-agent-subscription
     namespace: openshift-lifecycle-agent
   spec:
     channel: "stable"
     name: lifecycle-agent
     source: redhat-operators
     sourceNamespace: openshift-marketplace

   Copy to Clipboard Toggle word wrap

   1. 运行以下命令来创建 Subscription CR：

      $ oc create -f <subscription_filename>.yaml

      Copy to Clipboard Toggle word wrap

验证

1. 要验证安装是否成功，请运行以下命令来检查 CSV 资源：

   $ oc get csv -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   输出示例

   NAME                              DISPLAY                     VERSION               REPLACES                           PHASE
   lifecycle-agent.v4.17.0           Openshift Lifecycle Agent   4.17.0                Succeeded

   Copy to Clipboard Toggle word wrap

2. 运行以下命令验证 Lifecycle Agent 是否正在运行：

   $ oc get deploy -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   输出示例

   NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
   lifecycle-agent-controller-manager   1/1     1            1           14s

   Copy to Clipboard Toggle word wrap

流程

1. 在 OpenShift Container Platform Web 控制台中导航至 Operators OperatorHub。
2. 从可用的 Operator 列表中选择 Lifecycle Agent，然后点 Install。
3. 在 Install Operator 页面中，在 A specific namespace on the cluster 下选择 openshift-lifecycle-agent。
4. 点 Install。

验证

1. 确认安装成功：

   1. 点 Operators Installed Operators。

   2. 确保 openshift-lifecycle-agent 项目中列出的 Lifecycle Agent 的 Status 为 InstallSucceeded。

      注意

      在安装过程中，Operator 可能会显示 Failed 状态。如果安装过程结束后有 InstallSucceeded 信息，您可以忽略这个 Failed 信息。

1. 点 Operators Installed Operators，检查 Operator Subscriptions 和 Install Plans 选项卡中的 Status 是否有任何错误。
2. 点 Workloads Pods，在 openshift-lifecycle-agent 项目中检查 pod 的日志。

流程

1. 从 ztp-site-generate 容器镜像中提取以下 CR，并将它们推送到 source-cr 目录：

   LcaSubscriptionNS.yaml 文件示例

   apiVersion: v1
   kind: Namespace
   metadata:
     name: openshift-lifecycle-agent
     annotations:
       workload.openshift.io/allowed: management
       ran.openshift.io/ztp-deploy-wave: "2"
     labels:
       kubernetes.io/metadata.name: openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   LcaSubscriptionOperGroup.yaml 文件示例

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: lifecycle-agent-operatorgroup
     namespace: openshift-lifecycle-agent
     annotations:
       ran.openshift.io/ztp-deploy-wave: "2"
   spec:
     targetNamespaces:
       - openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   LcaSubscription.yaml 文件示例

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: lifecycle-agent
     namespace: openshift-lifecycle-agent
     annotations:
       ran.openshift.io/ztp-deploy-wave: "2"
   spec:
     channel: "stable"
     name: lifecycle-agent
     source: redhat-operators
     sourceNamespace: openshift-marketplace
     installPlanApproval: Manual
   status:
     state: AtLatestKnown

   Copy to Clipboard Toggle word wrap

   目录结构示例

   ├── kustomization.yaml
   ├── sno
   │   ├── example-cnf.yaml
   │   ├── common-ranGen.yaml
   │   ├── group-du-sno-ranGen.yaml
   │   ├── group-du-sno-validator-ranGen.yaml
   │   └── ns.yaml
   ├── source-crs
   │   ├── LcaSubscriptionNS.yaml
   │   ├── LcaSubscriptionOperGroup.yaml
   │   ├── LcaSubscription.yaml

   Copy to Clipboard Toggle word wrap

2. 将 CR 添加到通用 PolicyGenTemplate 中：

   apiVersion: ran.openshift.io/v1
   kind: PolicyGenTemplate
   metadata:
     name: "example-common-latest"
     namespace: "ztp-common"
   spec:
     bindingRules:
       common: "true"
       du-profile: "latest"
     sourceFiles:
       - fileName: LcaSubscriptionNS.yaml
         policyName: "subscriptions-policy"
       - fileName: LcaSubscriptionOperGroup.yaml
         policyName: "subscriptions-policy"
       - fileName: LcaSubscription.yaml
         policyName: "subscriptions-policy"
   [...]

   Copy to Clipboard Toggle word wrap

流程

1. 从 ztp-site-generate 容器镜像中提取以下 CR，并将它们推送到 source-cr 目录：

   OadpSubscriptionNS.yaml 文件示例

   apiVersion: v1
   kind: Namespace
   metadata:
     name: openshift-adp
     annotations:
       ran.openshift.io/ztp-deploy-wave: "2"
     labels:
       kubernetes.io/metadata.name: openshift-adp

   Copy to Clipboard Toggle word wrap

   OadpSubscriptionOperGroup.yaml 文件示例

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: redhat-oadp-operator
     namespace: openshift-adp
     annotations:
       ran.openshift.io/ztp-deploy-wave: "2"
   spec:
     targetNamespaces:
     - openshift-adp

   Copy to Clipboard Toggle word wrap

   OadpSubscription.yaml 文件示例

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: redhat-oadp-operator
     namespace: openshift-adp
     annotations:
       ran.openshift.io/ztp-deploy-wave: "2"
   spec:
     channel: stable-1.4
     name: redhat-oadp-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace
     installPlanApproval: Manual
   status:
     state: AtLatestKnown

   Copy to Clipboard Toggle word wrap

   OadpOperatorStatus.yaml 文件示例

   apiVersion: operators.coreos.com/v1
   kind: Operator
   metadata:
     name: redhat-oadp-operator.openshift-adp
     annotations:
       ran.openshift.io/ztp-deploy-wave: "2"
   status:
     components:
       refs:
       - kind: Subscription
         namespace: openshift-adp
         conditions:
         - type: CatalogSourcesUnhealthy
           status: "False"
       - kind: InstallPlan
         namespace: openshift-adp
         conditions:
         - type: Installed
           status: "True"
       - kind: ClusterServiceVersion
         namespace: openshift-adp
         conditions:
         - type: Succeeded
           status: "True"
           reason: InstallSucceeded

   Copy to Clipboard Toggle word wrap

   目录结构示例

   ├── kustomization.yaml
   ├── sno
   │   ├── example-cnf.yaml
   │   ├── common-ranGen.yaml
   │   ├── group-du-sno-ranGen.yaml
   │   ├── group-du-sno-validator-ranGen.yaml
   │   └── ns.yaml
   ├── source-crs
   │   ├── OadpSubscriptionNS.yaml
   │   ├── OadpSubscriptionOperGroup.yaml
   │   ├── OadpSubscription.yaml
   │   ├── OadpOperatorStatus.yaml

   Copy to Clipboard Toggle word wrap

2. 将 CR 添加到通用 PolicyGenTemplate 中：

   apiVersion: ran.openshift.io/v1
   kind: PolicyGenTemplate
   metadata:
     name: "example-common-latest"
     namespace: "ztp-common"
   spec:
     bindingRules:
       common: "true"
       du-profile: "latest"
     sourceFiles:
       - fileName: OadpSubscriptionNS.yaml
         policyName: "subscriptions-policy"
       - fileName: OadpSubscriptionOperGroup.yaml
         policyName: "subscriptions-policy"
       - fileName: OadpSubscription.yaml
         policyName: "subscriptions-policy"
       - fileName: OadpOperatorStatus.yaml
         policyName: "subscriptions-policy"
   [...]

   Copy to Clipboard Toggle word wrap

3. 只为目标集群创建 DataProtectionApplication CR 和 S3 secret：

   1. 从 ztp-site-generate 容器镜像中提取以下 CR，并将它们推送到 source-cr 目录：

      DataProtectionApplication.yaml 文件示例

      apiVersion: oadp.openshift.io/v1alpha1
      kind: DataProtectionApplication
      metadata:
        name: dataprotectionapplication
        namespace: openshift-adp
        annotations:
          ran.openshift.io/ztp-deploy-wave: "100"
      spec:
        configuration:
          restic:
            enable: false 

      1

      
          velero:
            defaultPlugins:
              - aws
              - openshift
            resourceTimeout: 10m
        backupLocations:
          - velero:
              config:
                profile: "default"
                region: minio
                s3Url: $url
                insecureSkipTLSVerify: "true"
                s3ForcePathStyle: "true"
              provider: aws
              default: true
              credential:
                key: cloud
                name: cloud-credentials
              objectStorage:
                bucket: $bucketName 

      2

      
                prefix: $prefixName 

      3

      
      status:
        conditions:
        - reason: Complete
          status: "True"
          type: Reconciled

      Copy to Clipboard Toggle word wrap

      1

      对于基于镜像的升级，spec.configuration.restic.enable 字段必须设置为 false，因为升级后持久性卷内容会被保留并重复使用。

      2 3

      bucket 定义在 S3 后端中创建的存储桶名称。前缀定义要在存储桶中自动创建的子目录的名称。bucket 和前缀的组合对于每个目标集群都必须是唯一的，以避免它们间的干扰。要确保每个目标集群的唯一存储目录，您可以使用 RHACM hub 模板功能，例如 prefix: {{hub .ManagedClusterName hub}}。

      OadpSecret.yaml 文件示例

      apiVersion: v1
      kind: Secret
      metadata:
        name: cloud-credentials
        namespace: openshift-adp
        annotations:
          ran.openshift.io/ztp-deploy-wave: "100"
      type: Opaque

      Copy to Clipboard Toggle word wrap

      OadpBackupStorageLocationStatus.yaml 文件示例

      apiVersion: velero.io/v1
      kind: BackupStorageLocation
      metadata:
        namespace: openshift-adp
        annotations:
          ran.openshift.io/ztp-deploy-wave: "100"
      status:
        phase: Available

      Copy to Clipboard Toggle word wrap

      OadpBackupStorageLocationStatus.yaml CR 验证 OADP 创建的备份存储位置的可用性。

   2. 使用覆盖将 CR 添加到站点 PolicyGenTemplate 中：

      apiVersion: ran.openshift.io/v1
      kind: PolicyGenTemplate
      metadata:
        name: "example-cnf"
        namespace: "ztp-site"
      spec:
        bindingRules:
          sites: "example-cnf"
          du-profile: "latest"
        mcp: "master"
        sourceFiles:
          ...
          - fileName: OadpSecret.yaml
            policyName: "config-policy"
            data:
              cloud: <your_credentials> 

      1

      
          - fileName: DataProtectionApplication.yaml
            policyName: "config-policy"
            spec:
              backupLocations:
                - velero:
                    config:
                      region: minio
                      s3Url: <your_S3_URL> 

      2

      
                      profile: "default"
                      insecureSkipTLSVerify: "true"
                      s3ForcePathStyle: "true"
                    provider: aws
                    default: true
                    credential:
                      key: cloud
                      name: cloud-credentials
                    objectStorage:
                      bucket: <your_bucket_name> 

      3

      
                      prefix: <cluster_name> 

      4

      
          - fileName: OadpBackupStorageLocationStatus.yaml
            policyName: "config-policy"

      Copy to Clipboard Toggle word wrap

      1

      为您的 S3 存储后端指定您的凭证。

      2

      指定 S3 兼容存储桶的 URL。

      3 4

      bucket 定义在 S3 后端中创建的存储桶名称。prefix 定义 bucket 中自动创建的子目录的名称。bucket 和 prefix 的组合对于每个目标集群都必须是唯一的，以避免它们间的干扰。要确保每个目标集群的唯一存储目录，您可以使用 RHACM hub 模板功能，例如 prefix: {{hub .ManagedClusterName hub}}。

1. 这个版本不支持双栈网络。
2. 如果在断开连接的环境中安装了 seed 集群，则目标集群还必须在断开连接的环境中安装。
3. 代理配置必须在 seed 和 target 集群中启用或禁用。但是，在集群中配置的代理服务器不必匹配。

流程

1. 从 hub 中分离受管集群，从 seed 集群中删除任何不能位于 seed 镜像中的任何 RHACM 特定资源：

   1. 运行以下命令手动分离 seed 集群：

      $ oc delete managedcluster sno-worker-example

      Copy to Clipboard Toggle word wrap
      1. 等待受管集群被删除。删除集群后，创建正确的 SeedGenerator CR。Lifecycle Agent 清理 RHACM 工件。

   2. 如果使用 GitOps ZTP，请通过从 kustomization.yaml 中删除 seed 集群的 SiteConfig CR 来分离集群：

      1. 如果您有一个引用多个 SiteConfig CR 的 kustomization.yaml 文件，请从 kustomization.yaml 中删除您看到的集群 SiteConfig CR：

         apiVersion: kustomize.config.k8s.io/v1beta1
         kind: Kustomization
         
         generators:
         #- example-seed-sno1.yaml
         - example-target-sno2.yaml
         - example-target-sno3.yaml

         Copy to Clipboard Toggle word wrap

      2. 如果您有一个引用一个 SiteConfig CR 的 kustomization.yaml，请从 kustomization.yaml 中删除您看到的集群 SiteConfig CR，并添加 generators: {} 行：

         apiVersion: kustomize.config.k8s.io/v1beta1
         kind: Kustomization
         
         generators: {}

         Copy to Clipboard Toggle word wrap

      3. 在 Git 存储库中提交 kustomization.yaml 更改，并将更改推送到您的存储库。

         ArgoCD 管道检测到更改并删除受管集群。

2. 创建 Secret 对象，以便您可以将 seed 镜像推送到 registry。

   1. 运行以下命令来创建身份验证文件：

      $ MY_USER=myuserid
      $ AUTHFILE=/tmp/my-auth.json
      $ podman login --authfile ${AUTHFILE} -u ${MY_USER} quay.io/${MY_USER}

      Copy to Clipboard Toggle word wrap

      $ base64 -w 0 ${AUTHFILE} ; echo

      Copy to Clipboard Toggle word wrap

   2. 将输出复制到 openshift-lifecycle-agent 命名空间中名为 seedgen 的 Secret YAML 文件中的 seedAuth 字段中：

      apiVersion: v1
      kind: Secret
      metadata:
        name: seedgen 

      1

      
        namespace: openshift-lifecycle-agent
      type: Opaque
      data:
        seedAuth: <encoded_AUTHFILE> 

      2

      Copy to Clipboard Toggle word wrap

      1

      Secret 资源必须具有 name: seedgen 和 namespace: openshift-lifecycle-agent 字段。

      2

      指定用于推送生成的 seed 镜像的 write-access 到 registry 的 base64 编码的 authfile。

   3. 运行以下命令来应用 Secret ：

      $ oc apply -f secretseedgenerator.yaml

      Copy to Clipboard Toggle word wrap

3. 创建 SeedGenerator CR：

   apiVersion: lca.openshift.io/v1
   kind: SeedGenerator
   metadata:
     name: seedimage 

   1

   
   spec:
     seedImage: <seed_container_image> 

   2

   Copy to Clipboard Toggle word wrap

   1

   SeedGenerator CR 必须命名为 seedimage。

   2

   指定容器镜像 URL，如 quay.io/example/seed-container-image:<tag>。建议您使用 <seed_cluster_name>:<ocp_version> 格式。

4. 运行以下命令来生成 seed 镜像：

   $ oc apply -f seedgenerator.yaml

   Copy to Clipboard Toggle word wrap
   重要

   当 Lifecycle Agent 生成 seed 镜像时，集群会重启并丢失 API 功能。应用 SeedGenerator CR 会停止 kubelet 和 CRI-O 操作，然后启动镜像生成。

流程

1. 为安装 OADP Operator 的同一命名空间中的平台工件 (openshift-adp) 创建 OADP Backup 和 Restore CR。

   1. 如果目标集群由 RHACM 管理，请添加以下 YAML 文件来备份和恢复 RHACM 工件：

      用于 RHACM 的 PlatformBackupRestore.yaml

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        name: acm-klusterlet
        annotations:
          lca.openshift.io/apply-label: "apps/v1/deployments/open-cluster-management-agent/klusterlet,v1/secrets/open-cluster-management-agent/bootstrap-hub-kubeconfig,rbac.authorization.k8s.io/v1/clusterroles/klusterlet,v1/serviceaccounts/open-cluster-management-agent/klusterlet,scheduling.k8s.io/v1/priorityclasses/klusterlet-critical,rbac.authorization.k8s.io/v1/clusterroles/open-cluster-management:klusterlet-admin-aggregate-clusterrole,rbac.authorization.k8s.io/v1/clusterrolebindings/klusterlet,operator.open-cluster-management.io/v1/klusterlets/klusterlet,apiextensions.k8s.io/v1/customresourcedefinitions/klusterlets.operator.open-cluster-management.io,v1/secrets/open-cluster-management-agent/open-cluster-management-image-pull-credentials" 

      1

      
        labels:
          velero.io/storage-location: default
        namespace: openshift-adp
      spec:
        includedNamespaces:
        - open-cluster-management-agent
        includedClusterScopedResources:
        - klusterlets.operator.open-cluster-management.io
        - clusterroles.rbac.authorization.k8s.io
        - clusterrolebindings.rbac.authorization.k8s.io
        - priorityclasses.scheduling.k8s.io
        includedNamespaceScopedResources:
        - deployments
        - serviceaccounts
        - secrets
        excludedNamespaceScopedResources: []
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: acm-klusterlet
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "1"
      spec:
        backupName:
          acm-klusterlet

      Copy to Clipboard Toggle word wrap

      1

      如果您的 multiclusterHub CR 没有定义 .spec.imagePullSecret，且 hub 集群中的 open-cluster-management-agent 命名空间中不存在 secret，请删除 v1/secrets/open-cluster-management-agent/open-cluster-management-image-pull-credentials。

   2. 如果您通过 LVM 存储在集群中创建了持久性卷，请为 LVM Storage 工件添加以下 YAML 文件：

      用于 LVM 存储的 PlatformBackupRestoreLvms.yaml

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        labels:
          velero.io/storage-location: default
        name: lvmcluster
        namespace: openshift-adp
      spec:
        includedNamespaces:
          - openshift-storage
        includedNamespaceScopedResources:
          - lvmclusters
          - lvmvolumegroups
          - lvmvolumegroupnodestatuses
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: lvmcluster
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "2" 

      1

      
      spec:
        backupName:
          lvmcluster

      Copy to Clipboard Toggle word wrap

      1

      lca.openshift.io/apply-wave 值必须小于应用程序 Restore CR 中指定的值。

2. 如果您需要在升级后恢复应用程序，在 openshift-adp 命名空间中为应用程序创建 OADP Backup 和 Restore CR。

   1. 在 openshift-adp 命名空间中为集群范围的应用程序工件创建 OADP CR。

      LSO 和 LVM Storage 的集群范围应用程序工件的 OADP CR 示例

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        annotations:
          lca.openshift.io/apply-label: "apiextensions.k8s.io/v1/customresourcedefinitions/test.example.com,security.openshift.io/v1/securitycontextconstraints/test,rbac.authorization.k8s.io/v1/clusterroles/test-role,rbac.authorization.k8s.io/v1/clusterrolebindings/system:openshift:scc:test" 

      1

      
        name: backup-app-cluster-resources
        labels:
          velero.io/storage-location: default
        namespace: openshift-adp
      spec:
        includedClusterScopedResources:
        - customresourcedefinitions
        - securitycontextconstraints
        - clusterrolebindings
        - clusterroles
        excludedClusterScopedResources:
        - Namespace
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: test-app-cluster-resources
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "3" 

      2

      
      spec:
        backupName:
          backup-app-cluster-resources

      Copy to Clipboard Toggle word wrap

      1

      将示例资源名称替换为您的实际资源。

      2

      lca.openshift.io/apply-wave 值必须高于平台 Restore CR 中的值，并低于应用程序命名空间范围 Restore CR 中的值。

   2. 为您的命名空间范围的应用程序工件创建 OADP CR。

      使用 LSO 时 OADP CR 命名空间范围的应用程序工件示例

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        labels:
          velero.io/storage-location: default
        name: backup-app
        namespace: openshift-adp
      spec:
        includedNamespaces:
        - test
        includedNamespaceScopedResources:
        - secrets
        - persistentvolumeclaims
        - deployments
        - statefulsets
        - configmaps
        - cronjobs
        - services
        - job
        - poddisruptionbudgets
        - <application_custom_resources> 

      1

      
        excludedClusterScopedResources:
        - persistentVolumes
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: test-app
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "4"
      spec:
        backupName:
          backup-app

      Copy to Clipboard Toggle word wrap

      1

      为应用程序定义自定义资源。

      使用 LVM 存储时 OADP CR 命名空间范围的应用程序工件示例

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        labels:
          velero.io/storage-location: default
        name: backup-app
        namespace: openshift-adp
      spec:
        includedNamespaces:
        - test
        includedNamespaceScopedResources:
        - secrets
        - persistentvolumeclaims
        - deployments
        - statefulsets
        - configmaps
        - cronjobs
        - services
        - job
        - poddisruptionbudgets
        - <application_custom_resources> 

      1

      
        includedClusterScopedResources:
        - persistentVolumes 

      2

      
        - logicalvolumes.topolvm.io 

      3

      
        - volumesnapshotcontents 

      4

      
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: test-app
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "4"
      spec:
        backupName:
          backup-app
        restorePVs: true
        restoreStatus:
          includedResources:
          - logicalvolumes 

      5

      Copy to Clipboard Toggle word wrap

      1

      为应用程序定义自定义资源。

      2

      必填字段。

      3

      必填字段

      4

      可选，如果使用 LVM 存储卷快照。

      5

      必填字段。

      重要

      同一版本的应用程序必须在 OpenShift Container Platform 的当前和目标发行版本中正常工作。

3. 运行以下命令，为 OADP CR 创建 ConfigMap 对象：

   $ oc create configmap oadp-cm-example --from-file=example-oadp-resources.yaml=<path_to_oadp_crs> -n openshift-adp

   Copy to Clipboard Toggle word wrap

4. 运行以下命令来修补 ImageBasedUpgrade CR：

   $ oc patch imagebasedupgrades.lca.openshift.io upgrade \
     -p='{"spec": {"oadpContent": [{"name": "oadp-cm-example", "namespace": "openshift-adp"}]}}' \
     --type=merge -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

流程

1. 创建包含额外清单的 YAML 文件，如 SR-IOV。

   SR-IOV 资源示例

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: "example-sriov-node-policy"
     namespace: openshift-sriov-network-operator
   spec:
     deviceType: vfio-pci
     isRdma: false
     nicSelector:
       pfNames: [ens1f0]
     nodeSelector:
       node-role.kubernetes.io/master: ""
     mtu: 1500
     numVfs: 8
     priority: 99
     resourceName: example-sriov-node-policy
   ---
   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: "example-sriov-network"
     namespace: openshift-sriov-network-operator
   spec:
     ipam: |-
       {
       }
     linkState: auto
     networkNamespace: sriov-namespace
     resourceName: example-sriov-node-policy
     spoofChk: "on"
     trust: "off"

   Copy to Clipboard Toggle word wrap

2. 运行以下命令来创建 ConfigMap 对象：

   $ oc create configmap example-extra-manifests-cm --from-file=example-extra-manifests.yaml=<path_to_extramanifest> -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

3. 运行以下命令来修补 ImageBasedUpgrade CR：

   $ oc patch imagebasedupgrades.lca.openshift.io upgrade \
     -p='{"spec": {"extraManifests": [{"name": "example-extra-manifests-cm", "namespace": "openshift-lifecycle-agent"}]}}' \
     --type=merge -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

流程

1. 创建包含 CatalogSource CR 的 YAML 文件：

   apiVersion: operators.coreos.com/v1
   kind: CatalogSource
   metadata:
     name: example-catalogsources
     namespace: openshift-marketplace
   spec:
     sourceType: grpc
     displayName: disconnected-redhat-operators
     image: quay.io/example-org/example-catalog:v1

   Copy to Clipboard Toggle word wrap

2. 运行以下命令来创建 ConfigMap 对象：

   $ oc create configmap example-catalogsources-cm --from-file=example-catalogsources.yaml=<path_to_catalogsource_cr> -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

3. 运行以下命令来修补 ImageBasedUpgrade CR：

   $ oc patch imagebasedupgrades.lca.openshift.io upgrade \
     -p='{"spec": {"extraManifests": [{"name": "example-catalogsources-cm", "namespace": "openshift-lifecycle-agent"}]}}' \
     --type=merge -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

流程

1. 确保用于 ArgoCD 策略应用程序的 Git 存储库包含以下目录结构：

   ├── source-crs/
   │   ├── ibu/
   │   │    ├── ImageBasedUpgrade.yaml
   │   │    ├── PlatformBackupRestore.yaml
   │   │    ├── PlatformBackupRestoreLvms.yaml
   │   │    ├── PlatformBackupRestoreWithIBGU.yaml
   ├── ...
   ├── kustomization.yaml

   Copy to Clipboard Toggle word wrap

   source-crs/ibu/PlatformBackupRestoreWithIBGU.yaml 文件在 ZTP 容器镜像中提供。

   PlatformBackupRestoreWithIBGU.yaml

   apiVersion: velero.io/v1
   kind: Backup
   metadata:
     name: acm-klusterlet
     annotations:
       lca.openshift.io/apply-label: "apps/v1/deployments/open-cluster-management-agent/klusterlet,v1/secrets/open-cluster-management-agent/bootstrap-hub-kubeconfig,rbac.authorization.k8s.io/v1/clusterroles/klusterlet,v1/serviceaccounts/open-cluster-management-agent/klusterlet,scheduling.k8s.io/v1/priorityclasses/klusterlet-critical,rbac.authorization.k8s.io/v1/clusterroles/open-cluster-management:klusterlet-work:ibu-role,rbac.authorization.k8s.io/v1/clusterroles/open-cluster-management:klusterlet-admin-aggregate-clusterrole,rbac.authorization.k8s.io/v1/clusterrolebindings/klusterlet,operator.open-cluster-management.io/v1/klusterlets/klusterlet,apiextensions.k8s.io/v1/customresourcedefinitions/klusterlets.operator.open-cluster-management.io,v1/secrets/open-cluster-management-agent/open-cluster-management-image-pull-credentials" 

   1

   
     labels:
       velero.io/storage-location: default
     namespace: openshift-adp
   spec:
     includedNamespaces:
     - open-cluster-management-agent
     includedClusterScopedResources:
     - klusterlets.operator.open-cluster-management.io
     - clusterroles.rbac.authorization.k8s.io
     - clusterrolebindings.rbac.authorization.k8s.io
     - priorityclasses.scheduling.k8s.io
     includedNamespaceScopedResources:
     - deployments
     - serviceaccounts
     - secrets
     excludedNamespaceScopedResources: []
   ---
   apiVersion: velero.io/v1
   kind: Restore
   metadata:
     name: acm-klusterlet
     namespace: openshift-adp
     labels:
       velero.io/storage-location: default
     annotations:
       lca.openshift.io/apply-wave: "1"
   spec:
     backupName:
       acm-klusterlet

   Copy to Clipboard Toggle word wrap

   1

   如果您的 multiclusterHub CR 没有定义 .spec.imagePullSecret，且 hub 集群中的 open-cluster-management-agent 命名空间中不存在 secret，请删除 v1/secrets/open-cluster-management-agent/open-cluster-management-image-pull-credentials。

   注意

   如果您直接在受管集群中执行基于镜像的升级，请使用 PlatformBackupRestore.yaml 文件。

   如果使用 LVM 存储来创建持久性卷，您可以使用 ZTP 容器镜像中提供的 source-crs/ibu/PlatformBackupRestoreLvms.yaml 来备份 LVM 存储资源。

   PlatformBackupRestoreLvms.yaml

   apiVersion: velero.io/v1
   kind: Backup
   metadata:
     labels:
       velero.io/storage-location: default
     name: lvmcluster
     namespace: openshift-adp
   spec:
     includedNamespaces:
       - openshift-storage
     includedNamespaceScopedResources:
       - lvmclusters
       - lvmvolumegroups
       - lvmvolumegroupnodestatuses
   ---
   apiVersion: velero.io/v1
   kind: Restore
   metadata:
     name: lvmcluster
     namespace: openshift-adp
     labels:
       velero.io/storage-location: default
     annotations:
       lca.openshift.io/apply-wave: "2" 

   1

   
   spec:
     backupName:
       lvmcluster

   Copy to Clipboard Toggle word wrap

   1

   lca.openshift.io/apply-wave 值必须小于应用程序 Restore CR 中指定的值。

2. 如果您需要在升级后恢复应用程序，在 openshift-adp 命名空间中为应用程序创建 OADP Backup 和 Restore CR：

   1. 在 openshift-adp 命名空间中为集群范围的应用程序工件创建 OADP CR：

      LSO 和 LVM Storage 的集群范围应用程序工件的 OADP CR 示例

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        annotations:
          lca.openshift.io/apply-label: "apiextensions.k8s.io/v1/customresourcedefinitions/test.example.com,security.openshift.io/v1/securitycontextconstraints/test,rbac.authorization.k8s.io/v1/clusterroles/test-role,rbac.authorization.k8s.io/v1/clusterrolebindings/system:openshift:scc:test" 

      1

      
        name: backup-app-cluster-resources
        labels:
          velero.io/storage-location: default
        namespace: openshift-adp
      spec:
        includedClusterScopedResources:
        - customresourcedefinitions
        - securitycontextconstraints
        - clusterrolebindings
        - clusterroles
        excludedClusterScopedResources:
        - Namespace
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: test-app-cluster-resources
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "3" 

      2

      
      spec:
        backupName:
          backup-app-cluster-resources

      Copy to Clipboard Toggle word wrap

      1

      将示例资源名称替换为您的实际资源。

      2

      lca.openshift.io/apply-wave 值必须高于平台 Restore CR 中的值，并低于应用程序命名空间范围 Restore CR 中的值。

   2. 在 source-crs/custom-crs 目录中为您的命名空间范围的应用程序工件创建 OADP CR：

      使用 LSO 时 OADP CR 命名空间范围的应用程序工件示例

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        labels:
          velero.io/storage-location: default
        name: backup-app
        namespace: openshift-adp
      spec:
        includedNamespaces:
        - test
        includedNamespaceScopedResources:
        - secrets
        - persistentvolumeclaims
        - deployments
        - statefulsets
        - configmaps
        - cronjobs
        - services
        - job
        - poddisruptionbudgets
        - <application_custom_resources> 

      1

      
        excludedClusterScopedResources:
        - persistentVolumes
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: test-app
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "4"
      spec:
        backupName:
          backup-app

      Copy to Clipboard Toggle word wrap

      1

      为应用程序定义自定义资源。

      使用 LVM 存储时 OADP CR 命名空间范围的应用程序工件示例

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        labels:
          velero.io/storage-location: default
        name: backup-app
        namespace: openshift-adp
      spec:
        includedNamespaces:
        - test
        includedNamespaceScopedResources:
        - secrets
        - persistentvolumeclaims
        - deployments
        - statefulsets
        - configmaps
        - cronjobs
        - services
        - job
        - poddisruptionbudgets
        - <application_custom_resources> 

      1

      
        includedClusterScopedResources:
        - persistentVolumes 

      2

      
        - logicalvolumes.topolvm.io 

      3

      
        - volumesnapshotcontents 

      4

      
      ---
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: test-app
        namespace: openshift-adp
        labels:
          velero.io/storage-location: default
        annotations:
          lca.openshift.io/apply-wave: "4"
      spec:
        backupName:
          backup-app
        restorePVs: true
        restoreStatus:
          includedResources:
          - logicalvolumes 

      5

      Copy to Clipboard Toggle word wrap

      1

      为应用程序定义自定义资源。

      2

      必填字段。

      3

      必填字段

      4

      可选，如果使用 LVM 存储卷快照。

      5

      必填字段。

      重要

      同一版本的应用程序必须在 OpenShift Container Platform 的当前和目标发行版本中正常工作。

3. 使用以下内容创建一个 kustomization.yaml ：

   apiVersion: kustomize.config.k8s.io/v1beta1
   kind: Kustomization
   
   configMapGenerator: 

   1

   
   - files:
     - source-crs/ibu/PlatformBackupRestoreWithIBGU.yaml
     #- source-crs/custom-crs/ApplicationClusterScopedBackupRestore.yaml
     #- source-crs/custom-crs/ApplicationApplicationBackupRestoreLso.yaml
     name: oadp-cm
     namespace: openshift-adp 

   2

   
   generatorOptions:
     disableNameSuffixHash: true

   Copy to Clipboard Toggle word wrap

   1

   使用 Backup 和 Restore CR 在 hub 集群中创建 oadp-cm ConfigMap 对象。

   2

   命名空间必须存在于所有受管集群上，以及用于生成并复制到集群的 OADP ConfigMap 的 hub。

4. 将更改推送到您的 Git 存储库。

流程

1. 使用现有站点 PolicyGenTemplate CR 中的 lca.openshift.io/target-ocp-version: <target_version> 标签所需的额外清单：

   apiVersion: ran.openshift.io/v1
   kind: PolicyGenTemplate
   metadata:
     name: example-sno
   spec:
     bindingRules:
       sites: "example-sno"
       du-profile: "4.15"
     mcp: "master"
     sourceFiles:
       - fileName: SriovNetwork.yaml
         policyName: "config-policy"
         metadata:
           name: "sriov-nw-du-fh"
           labels:
             lca.openshift.io/target-ocp-version: "4.15" 

   1

   
         spec:
           resourceName: du_fh
           vlan: 140
       - fileName: SriovNetworkNodePolicy.yaml
         policyName: "config-policy"
         metadata:
           name: "sriov-nnp-du-fh"
           labels:
             lca.openshift.io/target-ocp-version: "4.15"
         spec:
           deviceType: netdevice
           isRdma: false
           nicSelector:
             pfNames: ["ens5f0"]
           numVfs: 8
           priority: 10
           resourceName: du_fh
       - fileName: SriovNetwork.yaml
         policyName: "config-policy"
         metadata:
           name: "sriov-nw-du-mh"
           labels:
             lca.openshift.io/target-ocp-version: "4.15"
         spec:
           resourceName: du_mh
           vlan: 150
       - fileName: SriovNetworkNodePolicy.yaml
         policyName: "config-policy"
         metadata:
           name: "sriov-nnp-du-mh"
           labels:
             lca.openshift.io/target-ocp-version: "4.15"
         spec:
           deviceType: vfio-pci
           isRdma: false
           nicSelector:
             pfNames: ["ens7f0"]
           numVfs: 8
           priority: 10
           resourceName: du_mh
       - fileName: DefaultCatsrc.yaml 

   2

   
         policyName: "config-policy"
         metadata:
           name: default-cat-source
           namespace: openshift-marketplace
           labels:
               lca.openshift.io/target-ocp-version: "4.15"
         spec:
             displayName: default-cat-source
             image: quay.io/example-org/example-catalog:v1

   Copy to Clipboard Toggle word wrap

   1

   确保 lca.openshift.io/target-ocp-version 标签与 ImageBasedUpgrade CR 的 spec.seedImageRef.version 字段中指定的目标 OpenShift Container Platform 版本的 y-stream 或 z-stream 匹配。Lifecycle Agent 仅应用与指定版本匹配的 CR。

   2

   如果您不想使用自定义目录源，请删除此条目。

2. 将更改推送到您的 Git 存储库。

流程

1. 运行以下命令，将阈值增加到 65%：

   $ oc -n openshift-lifecycle-agent annotate ibu upgrade image-cleanup.lca.openshift.io/disk-usage-threshold-percent='65'

   Copy to Clipboard Toggle word wrap

2. （可选）运行以下命令来删除阈值覆盖：

   $ oc -n  openshift-lifecycle-agent annotate ibu upgrade image-cleanup.lca.openshift.io/disk-usage-threshold-percent-

   Copy to Clipboard Toggle word wrap

流程

1. 运行以下命令禁用自动镜像清理：

   $ oc -n openshift-lifecycle-agent annotate ibu upgrade image-cleanup.lca.openshift.io/on-prep='Disabled'

   Copy to Clipboard Toggle word wrap

2. （可选）运行以下命令来再次启用自动镜像清理：

   $ oc -n  openshift-lifecycle-agent annotate ibu upgrade image-cleanup.lca.openshift.io/on-prep-

   Copy to Clipboard Toggle word wrap