Chapter 2. Driver Toolkit

Procedure

1. The image URL of the driver-toolkit corresponding to a certain release can be extracted from the release image using the oc adm command:

   • For an x86 image, the command is as follows:

     $ oc adm release info quay.io/openshift-release-dev/ocp-release:{product-version}.z-x86_64 --image-for=driver-toolkit

   • For an ARM image, the command is as follows:

     $ oc adm release info quay.io/openshift-release-dev/ocp-release:{product-version}.z-aarch64 --image-for=driver-toolkit

   Example output

   The output for the ocp-release:4.11.0-x86_64 image is as follows:

   quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:514e256367e8707e2c30f18a4f06fbd6c821ab9776602d2488e861f577a357d0

2. This image can be pulled using a valid pull secret, such as the pull secret required to install OpenShift Container Platform.

   $ podman pull --authfile=path/to/pullsecret.json quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:<SHA>

1. The YAML defines an ImageStream for storing the simple-kmod driver container image, and a BuildConfig for building the container. Save this YAML as 0000-buildconfig.yaml.template.

   apiVersion: image.openshift.io/v1
   kind: ImageStream
   metadata:
     labels:
       app: simple-kmod-driver-container
     name: simple-kmod-driver-container
     namespace: simple-kmod-demo
   spec: {}
   ---
   apiVersion: build.openshift.io/v1
   kind: BuildConfig
   metadata:
     labels:
       app: simple-kmod-driver-build
     name: simple-kmod-driver-build
     namespace: simple-kmod-demo
   spec:
     nodeSelector:
       node-role.kubernetes.io/worker: ""
     runPolicy: "Serial"
     triggers:
       - type: "ConfigChange"
       - type: "ImageChange"
     source:
       git:
         ref: "master"
         uri: "https://github.com/openshift-psap/kvc-simple-kmod.git"
       type: Git
       dockerfile: |
         FROM DRIVER_TOOLKIT_IMAGE
   
         WORKDIR /build/
   
         # Expecting kmod software version as an input to the build
         ARG KMODVER
   
         # Grab the software from upstream
         RUN git clone https://github.com/openshift-psap/simple-kmod.git
         WORKDIR simple-kmod
   
         # Build and install the module
         RUN make all       KVER=$(rpm -q --qf "%{VERSION}-%{RELEASE}.%{ARCH}"  kernel-core) KMODVER=${KMODVER} \
         && make install   KVER=$(rpm -q --qf "%{VERSION}-%{RELEASE}.%{ARCH}"  kernel-core) KMODVER=${KMODVER}
   
         # Add the helper tools
         WORKDIR /root/kvc-simple-kmod
         ADD Makefile .
         ADD simple-kmod-lib.sh .
         ADD simple-kmod-wrapper.sh .
         ADD simple-kmod.conf .
         RUN mkdir -p /usr/lib/kvc/ \
         && mkdir -p /etc/kvc/ \
         && make install
   
         RUN systemctl enable kmods-via-containers@simple-kmod
     strategy:
       dockerStrategy:
         buildArgs:
           - name: KMODVER
             value: DEMO
     output:
       to:
         kind: ImageStreamTag
         name: simple-kmod-driver-container:demo

2. Substitute the correct driver toolkit image for the OpenShift Container Platform version you are running in place of “DRIVER_TOOLKIT_IMAGE” with the following commands.

   $ OCP_VERSION=$(oc get clusterversion/version -ojsonpath={.status.desired.version})

   $ DRIVER_TOOLKIT_IMAGE=$(oc adm release info $OCP_VERSION --image-for=driver-toolkit)

   $ sed "s#DRIVER_TOOLKIT_IMAGE#${DRIVER_TOOLKIT_IMAGE}#" 0000-buildconfig.yaml.template > 0000-buildconfig.yaml

3. Create the image stream and build config with

   $ oc create -f 0000-buildconfig.yaml

4. After the builder pod completes successfully, deploy the driver container image as a DaemonSet.

   1. The driver container must run with the privileged security context in order to load the kernel modules on the host. The following YAML file contains the RBAC rules and the DaemonSet for running the driver container. Save this YAML as 1000-drivercontainer.yaml.

      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: simple-kmod-driver-container
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: simple-kmod-driver-container
      rules:
      - apiGroups:
        - security.openshift.io
        resources:
        - securitycontextconstraints
        verbs:
        - use
        resourceNames:
        - privileged
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: simple-kmod-driver-container
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: simple-kmod-driver-container
      subjects:
      - kind: ServiceAccount
        name: simple-kmod-driver-container
      userNames:
      - system:serviceaccount:simple-kmod-demo:simple-kmod-driver-container
      ---
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        name: simple-kmod-driver-container
      spec:
        selector:
          matchLabels:
            app: simple-kmod-driver-container
        template:
          metadata:
            labels:
              app: simple-kmod-driver-container
          spec:
            serviceAccount: simple-kmod-driver-container
            serviceAccountName: simple-kmod-driver-container
            containers:
            - image: image-registry.openshift-image-registry.svc:5000/simple-kmod-demo/simple-kmod-driver-container:demo
              name: simple-kmod-driver-container
              imagePullPolicy: Always
              command: ["/sbin/init"]
              lifecycle:
                preStop:
                  exec:
                    command: ["/bin/sh", "-c", "systemctl stop kmods-via-containers@simple-kmod"]
              securityContext:
                privileged: true
            nodeSelector:
              node-role.kubernetes.io/worker: ""

   2. Create the RBAC rules and daemon set:

      $ oc create -f 1000-drivercontainer.yaml

5. After the pods are running on the worker nodes, verify that the simple_kmod kernel module is loaded successfully on the host machines with lsmod.

   1. Verify that the pods are running:

      $ oc get pod -n simple-kmod-demo

      Example output

      NAME                                 READY   STATUS      RESTARTS   AGE
      simple-kmod-driver-build-1-build     0/1     Completed   0          6m
      simple-kmod-driver-container-b22fd   1/1     Running     0          40s
      simple-kmod-driver-container-jz9vn   1/1     Running     0          40s
      simple-kmod-driver-container-p45cc   1/1     Running     0          40s

   2. Execute the lsmod command in the driver container pod:

      $ oc exec -it pod/simple-kmod-driver-container-p45cc -- lsmod | grep simple

      Example output

      simple_procfs_kmod     16384  0
      simple_kmod            16384  0