9.17.11.5. Managing mediated devices

Procedure

1. Identify the name selector and resource name values for the mediated devices by exploring a worker node:

   1. Start a debugging session with the worker node by using the oc debug command. For example:

      $ oc debug node/node-11.redhat.com

   2. Change the root directory of the shell process to the file system of the host node by running the following command:

      # chroot /host

   3. Navigate to the mdev_bus directory and view its contents. Each subdirectory name is a PCI address of a physical GPU. For example:

      # cd sys/class/mdev_bus && ls

      Example output:

      0000:4b:00.4

   4. Go to the directory for your physical device and list the supported mediated device types as defined by the hardware vendor. For example:

      # cd 0000:4b:00.4 && ls mdev_supported_types

      Example output:

      nvidia-742  nvidia-744	nvidia-746  nvidia-748	nvidia-750  nvidia-752
      nvidia-743  nvidia-745	nvidia-747  nvidia-749	nvidia-751  nvidia-753

   5. Select the mediated device type that you want to use and identify its name selector value by viewing the contents of its name file. For example:

      # cat nvidia-745/name

      Example output:

      NVIDIA A2-2Q

2. Open the HyperConverged CR in your default editor by running the following command:

   $ oc edit hyperconverged kubevirt-hyperconverged -n openshift-cnv

3. Create and expose the mediated devices by updating the configuration:

   1. Create mediated devices by adding them to the spec.mediatedDevicesConfiguration stanza.

   2. Expose the mediated devices to the cluster by adding the mdevNameSelector and resourceName values to the spec.permittedHostDevices.mediatedDevices stanza. The resourceName value is based on the mdevNameSelector value, but you use underscores instead of spaces.

      Example HyperConverged CR:

      apiVersion: hco.kubevirt.io/v1
      kind: HyperConverged
      metadata:
        name: kubevirt-hyperconverged
        namespace: openshift-cnv
      spec:
        mediatedDevicesConfiguration:
          mediatedDeviceTypes:
          - nvidia-745
          nodeMediatedDeviceTypes:
          - mediatedDeviceTypes:
            - nvidia-746
            nodeSelector:
              kubernetes.io/hostname: node-11.redhat.com
        permittedHostDevices:
          mediatedDevices:
          - mdevNameSelector: NVIDIA A2-2Q
            resourceName: nvidia.com/NVIDIA_A2-2Q
          - mdevNameSelector: NVIDIA A2-4Q
            resourceName: nvidia.com/NVIDIA_A2-4Q
      # ...

      where:

      mediatedDeviceTypes

      Specifies global settings for the cluster and is required.

      nodeMediatedDeviceTypes

      Specifies global configuration overrides for a specific node or group of nodes and is optional. Must be used with the global mediatedDeviceTypes configuration.

      mediatedDeviceTypes

      Specifies an override to the global mediatedDeviceTypes configuration for the specified nodes. Required if you use nodeMediatedDeviceTypes.

      nodeSelector

      Specifies the node selector and must include a key:value pair. Required if you use nodeMediatedDeviceTypes.

      mdevNameSelector

      Specifies the mediated devices that map to this value on the host.

      resourceName

      Specifies the matching resource name that is allocated on the node.

4. Save your changes and exit the editor.