Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Supported product and hardware configurations

Red Hat AI Inference Server 3.1

Supported hardware and software configurations for deploying Red Hat AI Inference Server

Red Hat AI Documentation Team

Abstract

Learn about supported hardware and software configurations for Red Hat AI Inference Server.

Preface
Copy link

This document describes the supported hardware, software, and delivery platforms that you can use to run Red Hat AI Inference Server in production environments.

Important

Technology Preview and Developer Preview features are provided for early access to potential new features.

Technology Preview or Developer Preview features are not supported or recommended for production workloads.

Chapter 1. Product and version compatibility
Copy link

The following table lists the supported product versions for Red Hat AI Inference Server 3.1.

Expand
Table 1.1. Product and version compatibility
ProductSupported version

Red Hat AI Inference Server

3.1

vLLM core

0.9.0.1

LLM Compressor

0.5.1 Technology Preview

Chapter 2. Supported AI accelerators
Copy link

The following tables list the supported AI accelerators for Red Hat AI Inference Server 3.1.

Important

Red Hat AI Inference Server 3.1 is not compatible with CUDA versions lower than 12.8.

Expand
Table 2.1. Supported NVIDIA AI accelerators
Container imagevLLM releaseAI acceleratorsRequirementsvLLM architecture supportLLM Compressor support

rhaiis/vllm‑cuda-rhel9

vLLM 0.9.0.1

  • Turing
  • Ampere
  • Ada
  • Hopper
  • Blackwell (compute capability 10.0)
  • x86
  • Aarch64 Developer Preview

x86 Technology Preview

Expand
Table 2.2. Supported AMD AI accelerators
Container imagevLLM releaseAI acceleratorsRequirementsvLLM architecture supportLLM Compressor support

rhaiis/vllm‑rocm-rhel9

vLLM 0.8.4

  • AMD Instinct MI210
  • AMD Instinct MI300X

x86

x86 Technology Preview

Expand
Table 2.3. Supported Google TPU AI accelerators
Container imagevLLM releaseAI acceleratorsRequirementsvLLM architecture supportLLM Compressor support

rhaiis/vllm‑xla-rhel9

vLLM 0.8.5

Google TPU v6e

x86 Developer Preview

Not supported

Chapter 3. Supported deployment environments
Copy link

The following deployment environments for Red Hat AI Inference Server are supported.

Expand
Table 3.1. Red Hat AI Inference Server supported deployment environments
EnvironmentSupported versionsDeployment notes

OpenShift Container Platform (self‑managed)

4.14 – 4.18

Deploy on bare‑metal hosts or virtual machines.

Red Hat OpenShift Service on AWS (ROSA)

4.14 – 4.18

Requires ROSA STS cluster with GPU‑enabled P5 or G5 node types.

Red Hat Enterprise Linux (RHEL)

9.2 – 10.0

Deploy on bare‑metal hosts or virtual machines.

Linux (not RHEL)

-

Supported under third‑party policy deployed on bare‑metal hosts or virtual machines. OpenShift Container Platform Operators are not required.

Kubernetes (not OpenShift Container Platform)

-

Supported under third‑party policy deployed on bare‑metal hosts or virtual machines.

Note

Red Hat AI Inference Server is available only as a container image. The host operating system and kernel must support the required accelerator drivers. For more information, see Supported AI accelerators.

The following table lists the OpenShift Container Platform software prerequisites for GPU deployments.

Expand
Table 4.1. Software prerequisites for GPU deployments
ComponentMinimum versionOperator

NVIDIA GPU Operator

24.3

NVIDIA GPU Operator OLM Operator

AMD GPU Operator

6.2

AMD GPU Operator OLM Operator

Node Feature Discovery [1]

4.14

Node Feature Discovery Operator

[1] Included by default with OpenShift Container Platform. Node Feature Discovery is required for scheduling NUMA-aware workloads.

Chapter 5. Lifecycle and update policy
Copy link

Security and critical bug fixes are delivered as container images available from the registry.access.redhat.com/rhaiis container registry and are announced through RHSA advisories. See RHAIIS container images on catalog.redhat.com for more details.

Legal Notice
Copy link

Copyright © 2025 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat