Chapter 3. Mirroring the required images for model inference

Procedure

1. Find the version of the following images that match your environment and pull the images with podman:

   • Node Feature Discovery (NFD) Operator
   • NVIDIA GPU Operator
   • AI Inference Server

2. Create an image set configuration custom resource (CR) that includes the NFD Operator, NVIDIA GPU Operator, and AI Inference Server images that you pulled in the previous step. For example, save the following ImageSetConfiguration CR as the file imageset-config.yaml:

   apiVersion: mirror.openshift.io/v2alpha1
   kind: ImageSetConfiguration
   mirror:
     operators:
     # Node Feature Discovery (NFD) Operator
     # Helps Openshift detect hardware capabilities like GPUs
     - catalog: registry.redhat.io/openshift4/ose-cluster-nfd-operator:latest
       packages:
         - name: nfd
           defaultChannel: stable
           channels:
             - name: stable
   
     # GPU Operator
     # Manages NVIDIA GPUs on OpenShift
     - catalog: registry.connect.redhat.com/nvidia/gpu-operator-bundle:latest
       packages:
         - name: gpu-operator-certified
           defaultChannel: stable
           channels:
             - name: stable
     additionalImages:
     # Red Hat AI Inference Server image
     - name: registry.redhat.io/rhaiis/vllm-cuda-rhel9:latest
     # Model image
     - name: registry.redhat.io/rhelai1/granite-3-1-8b-instruct-quantized-w8a8:1.5

3. Mirror the required images into the mirror registry. Run the following command:

   $ oc mirror --config imageset-config.yaml docker://<target_mirror_registry_url> --registry-config <path_to_pull_secret_json>

4. Alternatively, if you have already installed the NFD and NVIDIA GPU Operators in the cluster, create an ImageSetConfiguration CR that configures AI Inference Server and model images only:

   apiVersion: mirror.openshift.io/v2alpha1
   kind: ImageSetConfiguration
   mirror:
     additionalImages:
     - name: registry.redhat.io/rhaiis/vllm-cuda-rhel9:latest
     - registry.redhat.io/rhelai1/granite-3-1-8b-instruct-quantized-w8a8:1.5

5. Mirror the image set in the disconnected environment.
6. Configure the cluster for the mirror registry.