Chapter 2. Mirroring the Operator images for AI Inference Server and its dependencies

Procedure

1. Find the version of the following images that match your environment and pull the images with podman:

   • Node Feature Discovery (NFD) Operator
   • NVIDIA GPU Operator
   • AI Inference Server

2. Create an image set configuration custom resource (CR) that includes the NFD Operator, NVIDIA GPU Operator, and AI Inference Server images that you pulled in the previous step. For example, the following ImageSetConfiguration CR includes the image specs for AMD CPU architecture and CUDA accelerators:

   apiVersion: mirror.openshift.io/v2alpha1
   kind: ImageSetConfiguration
   mirror:
     operators:
     # Node Feature Discovery (NFD) Operator
     # Helps Openshift detect hardware capabilities like GPUs
     - catalog: registry.redhat.io/openshift4/ose-cluster-nfd-operator:latest
       packages:
         - name: nfd
           defaultChannel: stable
           channels:
             - name: stable
   
     # GPU Operator
     # Manages NVIDIA GPUs on OpenShift
     - catalog: registry.connect.redhat.com/nvidia/gpu-operator-bundle:latest
       packages:
         - name: gpu-operator-certified
           defaultChannel: stable
           channels:
             - name: stable
     additionalImages:
       # Red Hat AI Inference Server image
     - name: registry.redhat.io/rhaiis/vllm-cuda-rhel9:latest

   Copy to Clipboard Toggle word wrap

3. Alternatively, if you have already installed the NFD and NVIDIA GPU Operators in the cluster, create an ImageSetConfiguration CR that configures AI Inference Server only:

   apiVersion: mirror.openshift.io/v2alpha1
   kind: ImageSetConfiguration
   mirror:
     additionalImages:
     - name: registry.redhat.io/rhaiis/vllm-cuda-rhel9:latest

   Copy to Clipboard Toggle word wrap
4. Mirror the image set in the disconnected environment.
5. Configure the disconnected cluster to use the updated image set.