Red Hat Summit Red Hat SummitSupportKonsoleEntwicklerDemo startenKontakt

Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.

Chapter 3. Version 3.2.0 release notes

Red Hat AI Inference Server 3.2.0 release provides container images that optimizes inferencing with large language models (LLMs) for NVIDIA CUDA and AMD ROCm AI accelerators. The container images are available from registry.redhat.io:

  • registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.2.0
  • registry.redhat.io/rhaiis/vllm-rocm-rhel9:3.2.0

With Red Hat AI Inference Server, you can serve and inference models with higher performance, lower cost, and enterprise-grade stability and security. Red Hat AI Inference Server is built on the upstream, open source vLLM software project.

New versions of vLLM and LLM Compressor are included in this release:

The Red Hat AI Inference Server supported product and hardware configurations have been expanded. For more information, see Supported product and hardware configurations.

Expand
Table 3.1. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell support

Runs on NVIDIA B200 compute capability 10.0 GPUs with FP8 kernels and full CUDA Graph acceleration

NVIDIA Blackwell

FP8 KV-cache on ROCm

Roughly twice as large context windows with no accuracy loss

All AMD GPUs

Skinny GEMMs

Roughly 10% lower inference latency

AMD MI300X

Full CUDA Graph mode

6–8% improved average Time Per Output Token (TPOT) for small models.

NVIDIA A100 and H100

Auto FP16 fallback

Stable runs on pre-Ampere cards without manual flags, for example, NVIDIA T4 GPUs

Older NVIDIA GPUs

3.1. New models enabled
Link kopieren

Expand
Table 3.2. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell compute capability 12.0

Runs on NVIDIA RTX PRO 6000 Blackwell Server Edition supporting W8A8/FP8 kernels and related tuning

NVIDIA RTX PRO 6000 Blackwell Server Edition

ROCm improvements

Full‑graph capture for TritonAttention, quick All‑Reduce, and chunked pre‑fill

AMD ROCm

3.2. New models enabled
Link kopieren

Red Hat AI Inference Server 3.2.0 expands capabilities by enabling the following models added in vLLM v0.9.1:

  • LoRA support for InternVL
  • Magistral
  • Minicpm eagle support
  • NemotronH

The following models were added in vLLM v0.9.0:

  • dots1
  • Ernie 4.5
  • FalconH1
  • Gemma‑3
  • GLM‑4.1 V
  • GPT‑2 for Sequence Classification
  • Granite 4
  • Keye‑VL‑8B‑Preview
  • LlamaGuard4
  • MiMo-7B
  • MiniMax-M1
  • MiniMax-VL-01
  • Ovis 1.6, Ovis 2
  • Phi‑tiny‑MoE‑instruct
  • Qwen 3 Embedding & Reranker
  • Slim-MoE
  • Tarsier 2
  • Tencent HunYuan‑MoE‑V1

3.3. New developer features
Link kopieren

Improved scheduler performance
The vLLM scheduler API CachedRequestData class has been updated, resulting in improved performance for object and cached sampler‑ID stores.
CUDA graph execution
  • CUDA graph execution is now available for all FlashAttention-3 (FA3) and FlashMLA paths, including prefix‑caching.
  • New live CUDA graph capture progress bar makes debugging easier.
Scheduling
Priority scheduling is now implemented in the vLLM V1 engine.
Nach oben
Red Hat logoGithubredditYoutubeTwitter

Lernen

Testen, kaufen und verkaufen

Communitys

Über Red Hat Dokumentation

Wir helfen Red Hat Benutzern, mit unseren Produkten und Diensten innovativ zu sein und ihre Ziele zu erreichen – mit Inhalten, denen sie vertrauen können. Entdecken Sie unsere neuesten Updates.

Mehr Inklusion in Open Source

Red Hat hat sich verpflichtet, problematische Sprache in unserem Code, unserer Dokumentation und unseren Web-Eigenschaften zu ersetzen. Weitere Einzelheiten finden Sie in Red Hat Blog.

Über Red Hat

Wir liefern gehärtete Lösungen, die es Unternehmen leichter machen, plattform- und umgebungsübergreifend zu arbeiten, vom zentralen Rechenzentrum bis zum Netzwerkrand.

Theme

© 2025 Red Hat