Red Hat Summit Red Hat SummitSupportoConsoleSviluppatoriInizia una provaContatti

Questo contenuto non è disponibile nella lingua selezionata.

Chapter 4. Version 3.2.0 release notes

Red Hat AI Inference Server 3.2.0 release provides container images that optimizes inferencing with large language models (LLMs) for NVIDIA CUDA and AMD ROCm AI accelerators. The container images are available from registry.redhat.io:

  • registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.2.0
  • registry.redhat.io/rhaiis/vllm-rocm-rhel9:3.2.0

With Red Hat AI Inference Server, you can serve and inference models with higher performance, lower cost, and enterprise-grade stability and security. Red Hat AI Inference Server is built on the upstream, open source vLLM software project.

New versions of vLLM and LLM Compressor are included in this release:

The Red Hat AI Inference Server supported product and hardware configurations have been expanded. For more information, see Supported product and hardware configurations.

Expand
Table 4.1. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell support

Runs on NVIDIA B200 compute capability 10.0 GPUs with FP8 kernels and full CUDA Graph acceleration

NVIDIA Blackwell

FP8 KV-cache on ROCm

Roughly twice as large context windows with no accuracy loss

All AMD GPUs

Skinny GEMMs

Roughly 10% lower inference latency

AMD MI300X

Full CUDA Graph mode

6–8% improved average Time Per Output Token (TPOT) for small models.

NVIDIA A100 and H100

Auto FP16 fallback

Stable runs on pre-Ampere cards without manual flags, for example, NVIDIA T4 GPUs

Older NVIDIA GPUs

4.1. New models enabled
Copia collegamento

Expand
Table 4.2. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell compute capability 12.0

Runs on NVIDIA RTX PRO 6000 Blackwell Server Edition supporting W8A8/FP8 kernels and related tuning

NVIDIA RTX PRO 6000 Blackwell Server Edition

ROCm improvements

Full‑graph capture for TritonAttention, quick All‑Reduce, and chunked pre‑fill

AMD ROCm

4.2. New models enabled
Copia collegamento

Red Hat AI Inference Server 3.2.0 expands capabilities by enabling the following models added in vLLM v0.9.1:

  • LoRA support for InternVL
  • Magistral
  • Minicpm eagle support
  • NemotronH

The following models were added in vLLM v0.9.0:

  • dots1
  • Ernie 4.5
  • FalconH1
  • Gemma‑3
  • GLM‑4.1 V
  • GPT‑2 for Sequence Classification
  • Granite 4
  • Keye‑VL‑8B‑Preview
  • LlamaGuard4
  • MiMo-7B
  • MiniMax-M1
  • MiniMax-VL-01
  • Ovis 1.6, Ovis 2
  • Phi‑tiny‑MoE‑instruct
  • Qwen 3 Embedding & Reranker
  • Slim-MoE
  • Tarsier 2
  • Tencent HunYuan‑MoE‑V1

4.3. New developer features
Copia collegamento

Improved scheduler performance
The vLLM scheduler API CachedRequestData class has been updated, resulting in improved performance for object and cached sampler‑ID stores.
CUDA graph execution
  • CUDA graph execution is now available for all FlashAttention-3 (FA3) and FlashMLA paths, including prefix‑caching.
  • New live CUDA graph capture progress bar makes debugging easier.
Scheduling
Priority scheduling is now implemented in the vLLM V1 engine.
Torna in cima
Red Hat logoGithubredditYoutubeTwitter

Formazione

Prova, acquista e vendi

Community

Informazioni sulla documentazione di Red Hat

Aiutiamo gli utenti Red Hat a innovarsi e raggiungere i propri obiettivi con i nostri prodotti e servizi grazie a contenuti di cui possono fidarsi. Esplora i nostri ultimi aggiornamenti.

Rendiamo l’open source più inclusivo

Red Hat si impegna a sostituire il linguaggio problematico nel codice, nella documentazione e nelle proprietà web. Per maggiori dettagli, visita il Blog di Red Hat.

Informazioni su Red Hat

Forniamo soluzioni consolidate che rendono più semplice per le aziende lavorare su piattaforme e ambienti diversi, dal datacenter centrale all'edge della rete.

Theme

© 2025 Red Hat