专用硬件和驱动程序启用

流程

1. 可以使用 oc adm 命令从发行镜像中提取与特定发行版本对应的 driver-toolkit 镜像 URL：

   • 对于 x86 镜像，命令如下：

     $ oc adm release info quay.io/openshift-release-dev/ocp-release:{product-version}.z-x86_64 --image-for=driver-toolkit

   • 对于 ARM 镜像，命令如下：

     $ oc adm release info quay.io/openshift-release-dev/ocp-release:{product-version}.z-aarch64 --image-for=driver-toolkit

   输出示例

   ocp-release:4.11.0-x86_64 镜像的输出如下：

   quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:514e256367e8707e2c30f18a4f06fbd6c821ab9776602d2488e861f577a357d0

2. 可以使用有效的 pull secret 拉取此镜像，如安装 OpenShift Container Platform 所需的 pull secret。

   $ podman pull --authfile=path/to/pullsecret.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<SHA>

1. YAML 定义了 ImageStream，用于存储 simple-kmod 驱动程序容器镜像，以及用于构建容器的 BuildConfig。将此 YAML 保存为 0000-buildconfig.yaml.template。

   apiVersion: image.openshift.io/v1
   kind: ImageStream
   metadata:
     labels:
       app: simple-kmod-driver-container
     name: simple-kmod-driver-container
     namespace: simple-kmod-demo
   spec: {}
   ---
   apiVersion: build.openshift.io/v1
   kind: BuildConfig
   metadata:
     labels:
       app: simple-kmod-driver-build
     name: simple-kmod-driver-build
     namespace: simple-kmod-demo
   spec:
     nodeSelector:
       node-role.kubernetes.io/worker: ""
     runPolicy: "Serial"
     triggers:
       - type: "ConfigChange"
       - type: "ImageChange"
     source:
       git:
         ref: "master"
         uri: "https://github.com/openshift-psap/kvc-simple-kmod.git"
       type: Git
       dockerfile: |
         FROM DRIVER_TOOLKIT_IMAGE
   
         WORKDIR /build/
   
         # Expecting kmod software version as an input to the build
         ARG KMODVER
   
         # Grab the software from upstream
         RUN git clone https://github.com/openshift-psap/simple-kmod.git
         WORKDIR simple-kmod
   
         # Build and install the module
         RUN make all       KVER=$(rpm -q --qf "%{VERSION}-%{RELEASE}.%{ARCH}"  kernel-core) KMODVER=${KMODVER} \
         && make install   KVER=$(rpm -q --qf "%{VERSION}-%{RELEASE}.%{ARCH}"  kernel-core) KMODVER=${KMODVER}
   
         # Add the helper tools
         WORKDIR /root/kvc-simple-kmod
         ADD Makefile .
         ADD simple-kmod-lib.sh .
         ADD simple-kmod-wrapper.sh .
         ADD simple-kmod.conf .
         RUN mkdir -p /usr/lib/kvc/ \
         && mkdir -p /etc/kvc/ \
         && make install
   
         RUN systemctl enable kmods-via-containers@simple-kmod
     strategy:
       dockerStrategy:
         buildArgs:
           - name: KMODVER
             value: DEMO
     output:
       to:
         kind: ImageStreamTag
         name: simple-kmod-driver-container:demo

2. 在以下命令中，使用您运行的 OpenShift Container Platform 版本的相关的正确 driver toolki 镜像替换 "DRIVER_TOOLKIT_IMAGE" 部分。

   $ OCP_VERSION=$(oc get clusterversion/version -ojsonpath={.status.desired.version})

   $ DRIVER_TOOLKIT_IMAGE=$(oc adm release info $OCP_VERSION --image-for=driver-toolkit)

   $ sed "s#DRIVER_TOOLKIT_IMAGE#${DRIVER_TOOLKIT_IMAGE}#" 0000-buildconfig.yaml.template > 0000-buildconfig.yaml

3. 使用创建镜像流和构建配置

   $ oc create -f 0000-buildconfig.yaml

4. 构建器 Pod 成功完成后，将驱动程序容器镜像部署为 DaemonSet。

   1. 驱动程序容器必须使用特权安全上下文运行，才能在主机上加载内核模块。以下 YAML 文件包含用于运行驱动程序容器的 RBAC 规则和 DaemonSet。将此 YAML 保存为 1000-drivercontainer.yaml。

      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: simple-kmod-driver-container
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: simple-kmod-driver-container
      rules:
      - apiGroups:
        - security.openshift.io
        resources:
        - securitycontextconstraints
        verbs:
        - use
        resourceNames:
        - privileged
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: simple-kmod-driver-container
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: simple-kmod-driver-container
      subjects:
      - kind: ServiceAccount
        name: simple-kmod-driver-container
      userNames:
      - system:serviceaccount:simple-kmod-demo:simple-kmod-driver-container
      ---
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        name: simple-kmod-driver-container
      spec:
        selector:
          matchLabels:
            app: simple-kmod-driver-container
        template:
          metadata:
            labels:
              app: simple-kmod-driver-container
          spec:
            serviceAccount: simple-kmod-driver-container
            serviceAccountName: simple-kmod-driver-container
            containers:
            - image: image-registry.openshift-image-registry.svc:5000/simple-kmod-demo/simple-kmod-driver-container:demo
              name: simple-kmod-driver-container
              imagePullPolicy: Always
              command: ["/sbin/init"]
              lifecycle:
                preStop:
                  exec:
                    command: ["/bin/sh", "-c", "systemctl stop kmods-via-containers@simple-kmod"]
              securityContext:
                privileged: true
            nodeSelector:
              node-role.kubernetes.io/worker: ""

   2. 创建 RBAC 规则和守护进程集：

      $ oc create -f 1000-drivercontainer.yaml

5. 当 pod 在 worker 节点上运行后，使用 lsmod 验证在主机机器上是否成功载入了 simple_kmod 内核模块。

   1. 验证 pod 是否正在运行：

      $ oc get pod -n simple-kmod-demo

      输出示例

      NAME                                 READY   STATUS      RESTARTS   AGE
      simple-kmod-driver-build-1-build     0/1     Completed   0          6m
      simple-kmod-driver-container-b22fd   1/1     Running     0          40s
      simple-kmod-driver-container-jz9vn   1/1     Running     0          40s
      simple-kmod-driver-container-p45cc   1/1     Running     0          40s

   2. 在驱动程序容器 pod 中执行 lsmod 命令：

      $ oc exec -it pod/simple-kmod-driver-container-p45cc -- lsmod | grep simple

      输出示例

      simple_procfs_kmod     16384  0
      simple_kmod            16384  0

流程

1. 在 openshift-operators 命名空间中安装 SRO：

   1. 创建以下 Subscription CR，并将 YAML 保存到 sro-sub.yaml 文件中：

      Subscription CR 示例

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: openshift-special-resource-operator
        namespace: openshift-operators
      spec:
        channel: "stable"
        installPlanApproval: Automatic
        name: openshift-special-resource-operator
        source: redhat-operators
        sourceNamespace: openshift-marketplace

   2. 运行以下命令来创建订阅对象：

      $ oc create -f sro-sub.yaml

   3. 切换到 openshift-operators 项目：

      $ oc project openshift-operators

流程

1. 登陆到 OpenShift Container Platform Web 控制台。

2. 安装特殊资源 Operator：

   1. 在 OpenShift Container Platform Web 控制台中，点击 Operators → OperatorHub。
   2. 从可用的 Operator 列表中选择 Special Resource Operator，然后单击 Install。
   3. 在 Install Operator 页面中，选择集群上的一个特定命名空间，选择上一节中创建的命名空间，然后点 Install。

1. 进入到 Operators → Installed Operators 页面。

2. 确保 openshift-operators 项目中列出的 Special Resource Operator 的 Status 为 InstallSucceeded。

   注意

   在安装过程中，Operator 可能会显示 Failed 状态。如果安装过程结束后有 InstallSucceeded 信息，您可以忽略这个 Failed 信息。

3. 如果 Operator 没有被成功安装，请按照以下步骤进行故障排除：

   1. 导航到 Operators → Installed Operators 页面，检查 Operator Subscriptions 和 Install Plans 选项卡中的 Status 项中是否有任何错误。
   2. 进入到 Workloads → Pods 页面，检查 openshift-operators 项目中的 pod 的日志。

流程

1. 要创建 simple-kmod SpecialResource 对象，请定义用于构建镜像的镜像流和构建配置，以及用于运行容器的服务帐户、角色、角色绑定和守护进程集。需要服务帐户、角色和角色绑定来运行具有特权安全上下文的守护进程集，以便加载内核模块。

   1. 创建 templates 目录，并更改到此目录：

      $ mkdir -p chart/simple-kmod-0.0.1/templates

      $ cd chart/simple-kmod-0.0.1/templates

   2. 将镜像流和构建配置的 YAML 模板保存到 templates 目录中的 0000-buildconfig.yaml 中：

      apiVersion: image.openshift.io/v1
      kind: ImageStream
      metadata:
        labels:
          app: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}} 1
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}} 2
      spec: {}
      ---
      apiVersion: build.openshift.io/v1
      kind: BuildConfig
      metadata:
        labels:
          app: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverBuild}}  3
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverBuild}} 4
        annotations:
          specialresource.openshift.io/wait: "true"
          specialresource.openshift.io/driver-container-vendor: simple-kmod
          specialresource.openshift.io/kernel-affine: "true"
      spec:
        nodeSelector:
          node-role.kubernetes.io/worker: ""
        runPolicy: "Serial"
        triggers:
          - type: "ConfigChange"
          - type: "ImageChange"
        source:
          git:
            ref: {{.Values.specialresource.spec.driverContainer.source.git.ref}}
            uri: {{.Values.specialresource.spec.driverContainer.source.git.uri}}
          type: Git
        strategy:
          dockerStrategy:
            dockerfilePath: Dockerfile.SRO
            buildArgs:
              - name: "IMAGE"
                value: {{ .Values.driverToolkitImage  }}
              {{- range $arg := .Values.buildArgs }}
              - name: {{ $arg.name }}
                value: {{ $arg.value }}
              {{- end }}
              - name: KVER
                value: {{ .Values.kernelFullVersion }}
        output:
          to:
            kind: ImageStreamTag
            name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}:v{{.Values.kernelFullVersion}} 5

      1 2 3 4 5

      {{.Values.specialresource.metadata.name}} 等模板由 SRO 填写，具体基于 SpecialResource CR 中的字段和 Operator 已知的变量，如 {{.Values.KernelFullVersion}}。

   3. 将 templates 目录中的 RBAC 资源和守护进程设置的以下 YAML 模板保存为 1000-driver-container.yaml ：

      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
      rules:
      - apiGroups:
        - security.openshift.io
        resources:
        - securitycontextconstraints
        verbs:
        - use
        resourceNames:
        - privileged
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
      subjects:
      - kind: ServiceAccount
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
        namespace: {{.Values.specialresource.spec.namespace}}
      ---
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        labels:
          app: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
        name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
        annotations:
          specialresource.openshift.io/wait: "true"
          specialresource.openshift.io/state: "driver-container"
          specialresource.openshift.io/driver-container-vendor: simple-kmod
          specialresource.openshift.io/kernel-affine: "true"
          specialresource.openshift.io/from-configmap: "true"
      spec:
        updateStrategy:
          type: OnDelete
        selector:
          matchLabels:
            app: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
        template:
          metadata:
            labels:
              app: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
          spec:
            priorityClassName: system-node-critical
            serviceAccount: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
            serviceAccountName: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
            containers:
            - image: image-registry.openshift-image-registry.svc:5000/{{.Values.specialresource.spec.namespace}}/{{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}:v{{.Values.kernelFullVersion}}
              name: {{.Values.specialresource.metadata.name}}-{{.Values.groupName.driverContainer}}
              imagePullPolicy: Always
              command: ["/sbin/init"]
              lifecycle:
                preStop:
                  exec:
                    command: ["/bin/sh", "-c", "systemctl stop kmods-via-containers@{{.Values.specialresource.metadata.name}}"]
              securityContext:
                privileged: true
            nodeSelector:
              node-role.kubernetes.io/worker: ""
              feature.node.kubernetes.io/kernel-version.full: "{{.Values.KernelFullVersion}}"

   4. 进入 chart/simple-kmod-0.0.1 目录：

      $ cd ..

   5. 在 chart/simple-kmod-0.0.1 目录中，将 Chart 的以下 YAML 保存为 Chart.yaml ：

      apiVersion: v2
      name: simple-kmod
      description: Simple kmod will deploy a simple kmod driver-container
      icon: https://avatars.githubusercontent.com/u/55542927
      type: application
      version: 0.0.1
      appVersion: 1.0.0

2. 在 Chart 目录中，使用 helm package 命令创建 chart：

   $ helm package simple-kmod-0.0.1/

   输出示例

   Successfully packaged chart and saved it to: /data/<username>/git/<github_username>/special-resource-operator/yaml-for-docs/chart/simple-kmod-0.0.1/simple-kmod-0.0.1.tgz

3. 创建配置映射以存储 chart 文件：

   1. 为配置映射文件创建目录：

      $ mkdir cm

   2. 将 Helm Chart 复制到 cm 目录中：

      $ cp simple-kmod-0.0.1.tgz cm/simple-kmod-0.0.1.tgz

   3. 创建一个索引文件，指定包含 Helm Chart 的 Helm 仓库：

      $ helm repo index cm --url=cm://simple-kmod/simple-kmod-chart

   4. 为 Helm Chart 中定义的对象创建一个命名空间：

      $ oc create namespace simple-kmod

   5. 创建配置映射对象：

      $ oc create cm simple-kmod-chart --from-file=cm/index.yaml --from-file=cm/simple-kmod-0.0.1.tgz -n simple-kmod

4. 使用以下 SpecialResource 清单，使用您在配置映射中创建的 Helm Chart 部署 simple-kmod 对象。将此 YAML 保存为 simple-kmod-configmap.yaml ：

   apiVersion: sro.openshift.io/v1beta1
   kind: SpecialResource
   metadata:
     name: simple-kmod
   spec:
     #debug: true 1
     namespace: simple-kmod
     chart:
       name: simple-kmod
       version: 0.0.1
       repository:
         name: example
         url: cm://simple-kmod/simple-kmod-chart 2
     set:
       kind: Values
       apiVersion: sro.openshift.io/v1beta1
       kmodNames: ["simple-kmod", "simple-procfs-kmod"]
       buildArgs:
       - name: "KMODVER"
         value: "SRO"
     driverContainer:
       source:
         git:
           ref: "master"
           uri: "https://github.com/openshift-psap/kvc-simple-kmod.git"

   1

   可选：取消注释 #debug: true 行，使 chart 中的 YAML 文件完整显示在 Operator 日志中，并验证日志是否已正确创建并模板化。

   2

   spec.chart.repository.url 字段指示 SRO 在配置映射中查找 chart。

5. 在命令行中创建 SpecialResource 文件：

   $ oc create -f simple-kmod-configmap.yaml

1. 使用 oc get pods 命令显示构建 pod 的状态：

   $ oc get pods -n simple-kmod

   输出示例

   NAME                                                  READY   STATUS      RESTARTS   AGE
   simple-kmod-driver-build-12813789169ac0ee-1-build     0/1     Completed   0          7m12s
   simple-kmod-driver-container-12813789169ac0ee-mjsnh   1/1     Running     0          8m2s
   simple-kmod-driver-container-12813789169ac0ee-qtkff   1/1     Running     0          8m2s

2. 使用 oc logs 命令以及从上述 oc get pods 命令获取的构建 pod 名称，以显示 simple-kmod 驱动程序容器镜像构建的日志：

   $ oc logs pod/simple-kmod-driver-build-12813789169ac0ee-1-build -n simple-kmod

3. 要验证是否载入了 simple-kmod 内核模块，请在上面的 oc get pods 命令返回的一个驱动程序容器 pod 中执行 lsmod 命令：

   $ oc exec -n simple-kmod -it pod/simple-kmod-driver-container-12813789169ac0ee-mjsnh -- lsmod | grep simple

   输出示例

   simple_procfs_kmod     16384  0
   simple_kmod            16384  0

流程

1. 运行以下命令来创建 templates 目录：

   $ mkdir -p charts/acm-simple-kmod-0.0.1/templates

2. 运行以下命令来更改 templates 目录：

   $ cd charts/acm-simple-kmod-0.0.1/templates

3. 为 BuildConfig、Policy 和 PlacementRule 资源创建模板文件。

   1. 将镜像流和构建配置的 YAML 模板保存在 templates 目录中，存为 0001-buildconfig.yaml。

      apiVersion: build.openshift.io/v1
      kind: BuildConfig
      metadata:
          labels:
              app: {{ printf "%s-%s" .Values.specialResourceModule.metadata.name .Values.kernelFullVersion | replace "." "-" | replace "_" "-" | trunc 63 }}
          name: {{ printf "%s-%s" .Values.specialResourceModule.metadata.name .Values.kernelFullVersion | replace "." "-" | replace "_" "-" | trunc 63 }}
          annotations:
      specialresource.openshift.io/wait: "true"
      spec:
          nodeSelector:
              node-role.kubernetes.io/worker: ""
          runPolicy: "Serial"
          triggers:
              - type: "ConfigChange"
              - type: "ImageChange"
          source:
          dockerfile: |
              FROM {{ .Values.driverToolkitImage  }} as builder
              WORKDIR /build/
              RUN git clone -b {{.Values.specialResourceModule.spec.set.git.ref}} {{.Values.specialResourceModule.spec.set.git.uri}}
              WORKDIR /build/simple-kmod
              RUN make all install KVER={{ .Values.kernelFullVersion }}
              FROM registry.redhat.io/ubi8/ubi-minimal
              RUN microdnf -y install kmod
              COPY --from=builder /etc/driver-toolkit-release.json /etc/
              COPY --from=builder /lib/modules/{{ .Values.kernelFullVersion }}/* /lib/modules/{{ .Values.kernelFullVersion }}/
          strategy:
              dockerStrategy:
                  dockerfilePath: Dockerfile.SRO
                  buildArgs:
                      - name: "IMAGE"
                        value: {{ .Values.driverToolkitImage  }}
                      {{- range $arg := .Values.buildArgs }}
                      - name: {{ $arg.name }}
                        value: {{ $arg.value }}
                      {{- end }}
                      - name: KVER
                        value: {{ .Values.kernelFullVersion }}
          output:
              to:
                  kind: DockerImage
                  name: {{.Values.registry}}/{{.Values.specialResourceModule.metadata.name}}-{{.Values.groupName.driverContainer}}:{{.Values.kernelFullVersion}}

   2. 将 ACM 策略的 YAML 模板保存到 templates 目录中，存为 0002-policy.yaml。

      apiVersion: policy.open-cluster-management.io/v1
      kind: Policy
      metadata:
          name: policy-{{.Values.specialResourceModule.metadata.name}}-ds
          annotations:
              policy.open-cluster-management.io/categories: CM Configuration Management
              policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
              policy.open-cluster-management.io/standards: NIST-CSF
      spec:
          remediationAction: enforce
          disabled: false
          policy-templates:
              - objectDefinition:
                  apiVersion: policy.open-cluster-management.io/v1
                  kind: ConfigurationPolicy
                  metadata:
                      name: config-{{.Values.specialResourceModule.metadata.name}}-ds
                  spec:
                      remediationAction: enforce
                      severity: low
                      namespaceselector:
                          exclude:
                              - kube-*
                          include:
                              - '*'
                      object-templates:
                          - complianceType: musthave
                            objectDefinition:
                              apiVersion: v1
                              kind: Namespace
                              metadata:
                                  name: {{.Values.specialResourceModule.spec.namespace}}
                          - complianceType: mustonlyhave
                            objectDefinition:
                              apiVersion: v1
                              kind: ServiceAccount
                              metadata:
                                  name: {{.Values.specialResourceModule.metadata.name}}
                                  namespace: {{.Values.specialResourceModule.spec.namespace}}
                          - complianceType: mustonlyhave
                            objectDefinition:
                              apiVersion: rbac.authorization.k8s.io/v1
                              kind: Role
                              metadata:
                                  name: {{.Values.specialResourceModule.metadata.name}}
                                  namespace: {{.Values.specialResourceModule.spec.namespace}}
                              rules:
                              - apiGroups:
                                  - security.openshift.io
                                resources:
                                  - securitycontextconstraints
                                verbs:
                                  - use
                                resourceNames:
                                  - privileged
                          - complianceType: mustonlyhave
                            objectDefinition:
                              apiVersion: rbac.authorization.k8s.io/v1
                              kind: RoleBinding
                              metadata:
                                  name: {{.Values.specialResourceModule.metadata.name}}
                                  namespace: {{.Values.specialResourceModule.spec.namespace}}
                              roleRef:
                                  apiGroup: rbac.authorization.k8s.io
                                  kind: Role
                                  name: {{.Values.specialResourceModule.metadata.name}}
                              subjects:
                              - kind: ServiceAccount
                                name: {{.Values.specialResourceModule.metadata.name}}
                                namespace: {{.Values.specialResourceModule.spec.namespace}}
                          - complianceType: musthave
                            objectDefinition:
                              apiVersion: apps/v1
                              kind: DaemonSet
                              metadata:
                                  labels:
                                      app: {{ printf "%s-%s" .Values.specialResourceModule.metadata.name .Values.kernelFullVersion | replace "." "-" | replace "_" "-" | trunc 63 }}
                                  name: {{ printf "%s-%s" .Values.specialResourceModule.metadata.name .Values.kernelFullVersion | replace "." "-" | replace "_" "-" | trunc 63 }}
                                  namespace: {{.Values.specialResourceModule.spec.namespace}}
                              spec:
                                  updateStrategy:
                                      type: OnDelete
                                  selector:
                                      matchLabels:
                                          app: {{ printf "%s-%s" .Values.specialResourceModule.metadata.name .Values.kernelFullVersion | replace "." "-" | replace "_" "-" | trunc 63 }}
                                  template:
                                      metadata:
                                          labels:
                                              app: {{ printf "%s-%s" .Values.specialResourceModule.metadata.name .Values.kernelFullVersion | replace "." "-" | replace "_" "-" | trunc 63 }}
                                      spec:
                                          priorityClassName: system-node-critical
                                          serviceAccount: {{.Values.specialResourceModule.metadata.name}}
                                          serviceAccountName: {{.Values.specialResourceModule.metadata.name}}
                                          containers:
                                          - image: {{.Values.registry}}/{{.Values.specialResourceModule.metadata.name}}-{{.Values.groupName.driverContainer}}:{{.Values.kernelFullVersion}}
                                            name: {{.Values.specialResourceModule.metadata.name}}
                                            imagePullPolicy: Always
                                            command: [sleep, infinity]
                                            lifecycle:
                                              preStop:
                                                  exec:
                                                      command: ["modprobe", "-r", "-a" , "simple-kmod", "simple-procfs-kmod"]
                                            securityContext:
                                              privileged: true

   3. 将放置规则的此 YAML 模板保存到 templates 目录中，存为 0003-policy.yaml。

      apiVersion: apps.open-cluster-management.io/v1
      kind: PlacementRule
      metadata:
          name: {{.Values.specialResourceModule.metadata.name}}-placement
      spec:
          clusterConditions:
          - status: "True"
            type: ManagedClusterConditionAvailable
          clusterSelector:
            matchExpressions:
            - key: name
              operator: NotIn
              values:
              - local-cluster
      ---
      apiVersion: policy.open-cluster-management.io/v1
      kind: PlacementBinding
      metadata:
          name: {{.Values.specialResourceModule.metadata.name}}-binding
      placementRef:
          apiGroup: apps.open-cluster-management.io
          kind: PlacementRule
          name: {{.Values.specialResourceModule.metadata.name}}-placement
      subjects:
          - apiGroup: policy.open-cluster-management.io
            kind: Policy
            name: policy-{{.Values.specialResourceModule.metadata.name}}-ds

   4. 运行以下命令，进入 chart/acm-simple-kmod-0.0.1 目录：

      cd ..

   5. 在 charts/acm-simple-kmod-0.0.1 目录中将 chart 的以下 YAML 模板保存为 Chart.yaml ：

      apiVersion: v2
      name: acm-simple-kmod
      description: Build ACM enabled simple-kmod driver with SpecialResourceOperator
      icon: https://avatars.githubusercontent.com/u/55542927
      type: application
      version: 0.0.1
      appVersion: 1.6.4

4. 在 chart 目录中，使用以下命令创建 chart：

   $ helm package acm-simple-kmod-0.0.1/

   输出示例

   Successfully packaged chart and saved it to: <directory>/charts/acm-simple-kmod-0.0.1.tgz

5. 创建配置映射以存储 chart 文件。

   1. 运行以下命令，为配置映射文件创建一个目录：

      $ mkdir cm

   2. 运行以下命令，将 Helm Chart 复制到 cm 目录中：

      $ cp acm-simple-kmod-0.0.1.tgz cm/acm-simple-kmod-0.0.1.tgz

   3. 运行以下命令，创建指定包含 Helm chart 的 Helm 仓库的索引文件：

      $ helm repo index cm --url=cm://acm-simple-kmod/acm-simple-kmod-chart

   4. 运行以下命令，为 Helm chart 中定义的对象创建一个命名空间：

      $ oc create namespace acm-simple-kmod

   5. 运行以下命令来创建配置映射对象：

      $ oc create cm acm-simple-kmod-chart --from-file=cm/index.yaml --from-file=cm/acm-simple-kmod-0.0.1.tgz -n acm-simple-kmod

6. 使用以下 SpecialResourceModule 清单，使用配置映射中创建的 Helm chart 部署 simple-kmod 对象。将此 YAML 文件保存为 acm-simple-kmod.yaml ：

   apiVersion: sro.openshift.io/v1beta1
   kind: SpecialResourceModule
   metadata:
       name: acm-simple-kmod
   spec:
       namespace: acm-simple-kmod
       chart:
           name: acm-simple-kmod
           version: 0.0.1
           repository:
               name: acm-simple-kmod
               url: cm://acm-simple-kmod/acm-simple-kmod-chart
       set:
           kind: Values
           apiVersion: sro.openshift.io/v1beta1
           buildArgs:
               - name: "KMODVER"
                 value: "SRO"
           registry: <your_registry>  1
           git:
               ref: master
               uri: https://github.com/openshift-psap/kvc-simple-kmod.git
       watch:
               - path: "$.metadata.labels.openshiftVersion"
                 apiVersion: cluster.open-cluster-management.io/v1
                 kind: ManagedCluster
                 name: spoke1

   1

   指定您配置的 registry 的 URL。

7. 运行以下命令来创建特殊资源模块：

   $ oc apply -f charts/examples/acm-simple-kmod.yaml

验证

1. 运行以下命令，检查构建 pod 的状态：

   $ KUBECONFIG=~/hub/auth/kubeconfig oc get pod -n acm-simple-kmod

   输出示例

   NAME                                                   READY   STATUS      RESTARTS   AGE
   acm-simple-kmod-4-18-0-305-34-2-el8-4-x86-64-1-build   0/1     Completed   0          42m

2. 运行以下命令检查是否创建了策略：

   $ KUBECONFIG=~/hub/auth/kubeconfig oc get placementrules,placementbindings,policies -n acm-simple-kmod

   输出示例

   NAME                                                                      AGE   REPLICAS
   placementrule.apps.open-cluster-management.io/acm-simple-kmod-placement   40m
   
   NAME                                                                         AGE
   placementbinding.policy.open-cluster-management.io/acm-simple-kmod-binding   40m
   
   NAME                                                                 REMEDIATION ACTION   COMPLIANCE STATE   AGE
   policy.policy.open-cluster-management.io/policy-acm-simple-kmod-ds   enforce              Compliant          40m

3. 运行以下命令检查资源是否已协调：

   $ KUBECONFIG=~/hub/auth/kubeconfig oc get specialresourcemodule acm-simple-kmod -o json | jq -r '.status'

   输出示例

   {
     "versions": {
       "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6a3330ef5a178435721ff4efdde762261a9c55212e9b4534385e04037693fbe4": {
         "complete": true
       }
     }
   }

4. 运行以下命令，检查资源是否在 spoke 中运行：

   $ KUBECONFIG=~/spoke1/kubeconfig oc get ds,pod -n acm-simple-kmod

   输出示例

   AME                                                          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
   daemonset.apps/acm-simple-kmod-4-18-0-305-45-1-el8-4-x86-64   3         3         3       3            3           <none>          26m
   
   NAME                                                     READY   STATUS    RESTARTS   AGE
   pod/acm-simple-kmod-4-18-0-305-45-1-el8-4-x86-64-brw78   1/1     Running   0          26m
   pod/acm-simple-kmod-4-18-0-305-45-1-el8-4-x86-64-fqh5h   1/1     Running   0          26m
   pod/acm-simple-kmod-4-18-0-305-45-1-el8-4-x86-64-m9sfd   1/1     Running   0          26m

流程

1. 为 NFD Operator 创建命名空间。

   1. 创建定义 openshift-nfd 命名空间的以下 Namespace 自定义资源（CR），然后在 nfd-namespace.yaml 文件中保存 YAML：

      apiVersion: v1
      kind: Namespace
      metadata:
        name: openshift-nfd

   2. 运行以下命令创建命名空间：

      $ oc create -f nfd-namespace.yaml

2. 通过创建以下对象，在您上一步中创建的命名空间中安装 NFD Operator：

   1. 创建以下 OperatorGroup CR，并在 nfd -operatorgroup.yaml 文件中保存 YAML：

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        generateName: openshift-nfd-
        name: openshift-nfd
        namespace: openshift-nfd
      spec:
        targetNamespaces:
        - openshift-nfd

   2. 运行以下命令来创建 OperatorGroup CR:

      $ oc create -f nfd-operatorgroup.yaml

   3. 创建以下 Subscription CR，并将 YAML 保存到 nfd-sub.yaml 文件中：

      订阅示例

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: nfd
        namespace: openshift-nfd
      spec:
        channel: "stable"
        installPlanApproval: Automatic
        name: nfd
        source: redhat-operators
        sourceNamespace: openshift-marketplace

   4. 运行以下命令来创建订阅对象：

      $ oc create -f nfd-sub.yaml

   5. 进入 openshift-nfd 项目：

      $ oc project openshift-nfd

流程

1. 在 OpenShift Container Platform Web 控制台中，点击 Operators → OperatorHub。
2. 从可用的 Operator 列表中选择 Node Feature Discovery，然后点 Install。
3. 在 Install Operator 页面中，选择 A specific namespace on the cluster，然后点 Install。您不需要创建命名空间，因为它已为您创建。

1. 进入到 Operators → Installed Operators 页面。

2. 确保 openshift-nfd 项目中列出了 Node Feature Discovery，Status 为 InstallSucceeded。

   注意

   在安装过程中，Operator 可能会显示 Failed 状态。如果安装过程结束后有 InstallSucceeded 信息，您可以忽略这个 Failed 信息。

1. 导航到 Operators → Installed Operators 页面，检查 Operator Subscriptions 和 Install Plans 选项卡中的 Status 项中是否有任何错误。
2. 导航到 Workloads → Pods 页面，在 openshift-nfd 项目中检查 pod 的日志。

流程

1. 创建以下 NodeFeatureDiscovery 自定义资源（CR），然后在 NodeFeatureDiscovery.yaml 文件中保存 YAML：

   apiVersion: nfd.openshift.io/v1
   kind: NodeFeatureDiscovery
   metadata:
     name: nfd-instance
     namespace: openshift-nfd
   spec:
     instance: "" # instance is empty by default
     topologyupdater: false # False by default
     operand:
       image: registry.redhat.io/openshift4/ose-node-feature-discovery:v4.11
       imagePullPolicy: Always
     workerConfig:
       configData: |
         core:
         #  labelWhiteList:
         #  noPublish: false
           sleepInterval: 60s
         #  sources: [all]
         #  klog:
         #    addDirHeader: false
         #    alsologtostderr: false
         #    logBacktraceAt:
         #    logtostderr: true
         #    skipHeaders: false
         #    stderrthreshold: 2
         #    v: 0
         #    vmodule:
         ##   NOTE: the following options are not dynamically run-time configurable
         ##         and require a nfd-worker restart to take effect after being changed
         #    logDir:
         #    logFile:
         #    logFileMaxSize: 1800
         #    skipLogHeaders: false
         sources:
           cpu:
             cpuid:
         #     NOTE: whitelist has priority over blacklist
               attributeBlacklist:
                 - "BMI1"
                 - "BMI2"
                 - "CLMUL"
                 - "CMOV"
                 - "CX16"
                 - "ERMS"
                 - "F16C"
                 - "HTT"
                 - "LZCNT"
                 - "MMX"
                 - "MMXEXT"
                 - "NX"
                 - "POPCNT"
                 - "RDRAND"
                 - "RDSEED"
                 - "RDTSCP"
                 - "SGX"
                 - "SSE"
                 - "SSE2"
                 - "SSE3"
                 - "SSE4.1"
                 - "SSE4.2"
                 - "SSSE3"
               attributeWhitelist:
           kernel:
             kconfigFile: "/path/to/kconfig"
             configOpts:
               - "NO_HZ"
               - "X86"
               - "DMI"
           pci:
             deviceClassWhitelist:
               - "0200"
               - "03"
               - "12"
             deviceLabelFields:
               - "class"
     customConfig:
       configData: |
             - name: "more.kernel.features"
               matchOn:
               - loadedKMod: ["example_kmod3"]

1. 运行以下命令来创建 NodeFeatureDiscovery CR 实例：

   $ oc create -f NodeFeatureDiscovery.yaml

流程

1. 进入到 Operators → Installed Operators 页面。
2. 查找 Node Feature Discovery，并在 Provided APIs 下看到一个方框。
3. 单击 Create instance。
4. 编辑 NodeFeatureDiscovery CR 的值。
5. 点 Create。