Chapter 7. Configuring additional persistent storage for the RHEL AI instance

Procedure

1. Review the disks attached to the instance:

   $ lsblk

2. Create a DISK variable for the newly provisioned disk path. For example:

   $ DISK=/dev/vdb

3. Configure the system to automatically mount the disk to the /mnt directory. Add an entry to /etc/fstab:

   $ echo "${DISK} /mnt xfs defaults 0 0" | sudo tee -a /etc/fstab

   Note

   Replace xfs with your disk’s file system type if different (for example, ext4).

4. Reload the systemd service:

   $ sudo systemctl daemon-reload

5. Mount the disk:

   $ sudo mount -a

6. Create folders for Red Hat AI Inference data on the mounted disk:

   $ sudo mkdir -p /mnt/rhaiis/cache /mnt/rhaiis/models /mnt/rhaiis/tmp /mnt/rhaiis/config /mnt/vllm /mnt/flashinfer

7. Grant appropriate permissions to the mounted directory:

   $ sudo chmod u+rwX /mnt/

   $ sudo chown -R 1001:root /mnt/

8. If the configuration file does not already exist, create it by copying the example file:

   $ sudo cp /etc/containers/systemd/rhaiis.container.d/install.conf.example /etc/containers/systemd/rhaiis.container.d/install.conf

9. Configure Red Hat AI Inference to use the additional storage by editing the configuration file:

   $ sudo vi /etc/containers/systemd/rhaiis.container.d/install.conf

   Add the following environment variables to the [Container] section:

   [Container]
   # Configure cache locations to use the additional storage disk
   Environment=VLLM_CACHE_ROOT=/mnt/vllm
   Environment=FLASHINFER_WORKSPACE_BASE=/mnt/flashinfer

   Expand

   Table 7.1. Storage configuration parameters

   Parameter	Description
   VLLM_CACHE_ROOT	Location where vLLM stores compiled kernels and other cache files.
   FLASHINFER_WORKSPACE_BASE	Location where FlashInfer stores JIT-compiled kernel workspaces.

   Show more

10. Update the volume mounts in the Red Hat AI Inference container configuration to include the additional storage paths. Edit the main container file:

    $ sudo vi /etc/containers/systemd/rhaiis.container

    Replace the existing Volume lines with the following updated volume mounts that point to the additional storage disk. Delete the following existing lines:

    Volume=/var/lib/rhaiis/cache:/opt/app-root/src/.cache:Z
    Volume=/var/lib/rhaiis/models:/opt/app-root/src/models:Z
    Volume=/var/lib/rhaiis/tmp:/tmp:Z
    Volume=/var/lib/rhaiis/config:/opt/app-root/src/.config:Z

    Replace them with:

    Volume=/mnt/rhaiis/cache:/opt/app-root/src/.cache:Z
    Volume=/mnt/rhaiis/models:/opt/app-root/src/models:Z
    Volume=/mnt/rhaiis/tmp:/tmp:Z
    Volume=/mnt/rhaiis/config:/opt/app-root/src/.config:Z
    Volume=/mnt/vllm:/mnt/vllm:Z
    Volume=/mnt/flashinfer:/mnt/flashinfer:Z

11. Reload the systemd configuration:

    $ sudo systemctl daemon-reload

12. Restart the Red Hat AI Inference service to apply the changes:

    $ sudo systemctl restart rhaiis