Chapter 4. Intel Gaudi AI Accelerator integration

Procedure

1. Install the latest version of the Intel Gaudi Base Operator, as described in Intel Gaudi Base Operator OpenShift installation.

2. By default, OpenShift sets a per-pod PID limit of 4096. If your workload requires more processing power, such as when you use multiple Gaudi accelerators or when using vLLM with Ray, you must manually increase the per-pod PID limit to avoid Resource temporarily unavailable errors. These errors occur due to PID exhaustion. Red Hat recommends setting this limit to 32768, although values over 20000 are sufficient.

   1. Run the following command to label the node:

      oc label node <node_name> custom-kubelet=set-pod-pid-limit-kubelet

      Copy to Clipboard Toggle word wrap
   2. Optional: To prevent workload distribution on the affected node, you can mark the node as unschedulable and then drain it in preparation for maintenance. For more information, see Understanding how to evacuate pods on nodes.

   3. Create a custom-kubelet-pidslimit.yaml KubeletConfig resource file:

      oc create -f custom-kubelet-pidslimit.yaml

      Copy to Clipboard Toggle word wrap

   4. Populate the file with the following YAML code. Set the PodPidsLimit value to 32768:

      apiVersion: machineconfiguration.openshift.io/v1
      kind: KubeletConfig
      metadata:
        name: custom-kubelet-pidslimit
      spec:
        kubeletConfig:
          PodPidsLimit: 32768
        machineConfigPoolSelector:
          matchLabels:
            custom-kubelet: set-pod-pid-limit-kubelet

      Copy to Clipboard Toggle word wrap

   5. Apply the configuration:

      oc apply -f custom-kubelet-pidslimit.yaml

      Copy to Clipboard Toggle word wrap

      This operation causes the node to reboot. For more information, see Understanding node rebooting.

   6. Optional: If you previously marked the node as unschedulable, you can allow scheduling again after the node reboots.
3. Create a custom workbench image for Intel Gaudi AI accelerators, as described in Creating custom workbench images.

4. After installing the Intel Gaudi Base Operator, create an accelerator profile, as described in Working with accelerator profiles.

   Important

   By default, hardware profiles are hidden in the dashboard navigation menu and user interface, while accelerator profiles remain visible. In addition, user interface components associated with the deprecated accelerator profiles functionality are still displayed. To show the Settings Hardware profiles option in the dashboard navigation menu, and the user interface components associated with hardware profiles, set the disableHardwareProfiles value to false in the OdhDashboardConfig custom resource (CR) in OpenShift. For more information about setting dashboard configuration options, see Customizing the dashboard.