Red Hat SummitEste contenido no está disponible en el idioma seleccionado.
Supported product and hardware configurations
Supported hardware and software configurations for deploying Red Hat AI Inference Server
Abstract
Preface
This document describes the supported hardware, software, and delivery platforms that you can use to run Red Hat AI Inference Server in production environments.
Technology Preview and Developer Preview features are provided for early access to potential new features.
Technology Preview or Developer Preview features are not supported or recommended for production workloads.
Chapter 1. Product and version compatibility
The following table lists the supported product versions for Red Hat AI Inference Server 3.2.
| Red Hat AI Inference Server version | vLLM core version | LLM Compressor version |
|---|---|---|
| 3.2.2 | v0.10.1.1 | v0.7.1 |
| 3.2.1 | v0.10.0 | Not included in this release |
| 3.2.0 | v0.9.2 | Not included in this release |
Chapter 2. Supported AI accelerators
The following tables list the supported AI data center grade accelerators for Red Hat AI Inference Server 3.2.
Red Hat AI Inference Server only supports data center grade accelerators.
Red Hat AI Inference Server 3.2 is not compatible with CUDA versions lower than 12.8.
| Container image | vLLM release | AI accelerators | Requirements | vLLM architecture support | LLM Compressor support |
|---|---|---|---|---|---|
|
| vLLM v0.10.1.1 | NVIDIA data center GPUs:
|
| Not included by default |
NVIDIA T4 and A100 accelerators do not support FP8 (W8A8) quantization.
| Container image | vLLM release | AI accelerators | Requirements | vLLM architecture support | LLM Compressor support |
|---|---|---|---|---|---|
|
| vLLM v0.10.1.1 |
| x86 | x86 Technology Preview |
AMD GPUs support FP8 (W8A8) and GGUF quantization schemes only.
| Container image | vLLM release | AI accelerators | Requirements | vLLM architecture support | LLM Compressor support |
|---|---|---|---|---|---|
|
| vLLM v0.10.1.1 | Google v4, v5e, v5p, v6e, Trillium | x86 Developer Preview | Not supported |
| Container image | vLLM release | AI accelerators | Requirements | vLLM architecture support | LLM Compressor support |
|---|---|---|---|---|---|
|
| vLLM v0.10.1.1 | IBM Spyre | x86 Developer Preview | Not supported |
Chapter 3. Supported deployment environments
The following deployment environments for Red Hat AI Inference Server are supported.
| Environment | Supported versions | Deployment notes |
|---|---|---|
| OpenShift Container Platform (self‑managed) | 4.14 – 4.19 | Deploy on bare‑metal hosts or virtual machines. |
| Red Hat OpenShift Service on AWS (ROSA) | 4.14 – 4.19 | Requires a ROSA cluster with STS and GPU‑enabled P5 or G5 node types. See Prepare your environment for more information. |
| Red Hat Enterprise Linux (RHEL) | 9.2 – 10.0 | Deploy on bare‑metal hosts or virtual machines. |
| Linux (not RHEL) | - | Supported under third‑party policy deployed on bare‑metal hosts or virtual machines. OpenShift Container Platform Operators are not required. |
| Kubernetes (not OpenShift Container Platform) | - | Supported under third‑party policy deployed on bare‑metal hosts or virtual machines. |
Red Hat AI Inference Server is available only as a container image. The host operating system and kernel must support the required accelerator drivers. For more information, see Supported AI accelerators.
Chapter 4. OpenShift Container Platform software prerequisites for GPU deployments
The following table lists the OpenShift Container Platform software prerequisites for GPU deployments.
| Component | Minimum version | Operator |
|---|---|---|
| NVIDIA GPU Operator | 24.3 | |
| AMD GPU Operator | 6.2 | |
| Node Feature Discovery [1] | 4.14 |
[1] Included by default with OpenShift Container Platform. Node Feature Discovery is required for scheduling NUMA-aware workloads.
Chapter 5. Lifecycle and update policy
Security and critical bug fixes are delivered as container images available from the registry.access.redhat.com/rhaiis container registry and are announced through RHSA advisories. See RHAIIS container images on catalog.redhat.com for more details.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}