Red Hat Summit Red Hat SummitApoyoConsolaDesarrolladoresIniciar una pruebaContacto

Este contenido no está disponible en el idioma seleccionado.

Chapter 4. Version 3.2.0 release notes

Red Hat AI Inference Server 3.2.0 release provides container images that optimizes inferencing with large language models (LLMs) for NVIDIA CUDA and AMD ROCm AI accelerators. The container images are available from registry.redhat.io:

  • registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.2.0
  • registry.redhat.io/rhaiis/vllm-rocm-rhel9:3.2.0

With Red Hat AI Inference Server, you can serve and inference models with higher performance, lower cost, and enterprise-grade stability and security. Red Hat AI Inference Server is built on the upstream, open source vLLM software project.

New versions of vLLM and LLM Compressor are included in this release:

The Red Hat AI Inference Server supported product and hardware configurations have been expanded. For more information, see Supported product and hardware configurations.

Expand
Table 4.1. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell support

Runs on NVIDIA B200 compute capability 10.0 GPUs with FP8 kernels and full CUDA Graph acceleration

NVIDIA Blackwell

FP8 KV-cache on ROCm

Roughly twice as large context windows with no accuracy loss

All AMD GPUs

Skinny GEMMs

Roughly 10% lower inference latency

AMD MI300X

Full CUDA Graph mode

6–8% improved average Time Per Output Token (TPOT) for small models.

NVIDIA A100 and H100

Auto FP16 fallback

Stable runs on pre-Ampere cards without manual flags, for example, NVIDIA T4 GPUs

Older NVIDIA GPUs

4.1. New models enabled
Copiar enlace

Expand
Table 4.2. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell compute capability 12.0

Runs on NVIDIA RTX PRO 6000 Blackwell Server Edition supporting W8A8/FP8 kernels and related tuning

NVIDIA RTX PRO 6000 Blackwell Server Edition

ROCm improvements

Full‑graph capture for TritonAttention, quick All‑Reduce, and chunked pre‑fill

AMD ROCm

4.2. New models enabled
Copiar enlace

Red Hat AI Inference Server 3.2.0 expands capabilities by enabling the following models added in vLLM v0.9.1:

  • LoRA support for InternVL
  • Magistral
  • Minicpm eagle support
  • NemotronH

The following models were added in vLLM v0.9.0:

  • dots1
  • Ernie 4.5
  • FalconH1
  • Gemma‑3
  • GLM‑4.1 V
  • GPT‑2 for Sequence Classification
  • Granite 4
  • Keye‑VL‑8B‑Preview
  • LlamaGuard4
  • MiMo-7B
  • MiniMax-M1
  • MiniMax-VL-01
  • Ovis 1.6, Ovis 2
  • Phi‑tiny‑MoE‑instruct
  • Qwen 3 Embedding & Reranker
  • Slim-MoE
  • Tarsier 2
  • Tencent HunYuan‑MoE‑V1

4.3. New developer features
Copiar enlace

Improved scheduler performance
The vLLM scheduler API CachedRequestData class has been updated, resulting in improved performance for object and cached sampler‑ID stores.
CUDA graph execution
  • CUDA graph execution is now available for all FlashAttention-3 (FA3) and FlashMLA paths, including prefix‑caching.
  • New live CUDA graph capture progress bar makes debugging easier.
Scheduling
Priority scheduling is now implemented in the vLLM V1 engine.
Volver arriba
Red Hat logoGithubredditYoutubeTwitter

Aprender

Pruebe, compre y venda

Comunidades

Acerca de la documentación de Red Hat

Ayudamos a los usuarios de Red Hat a innovar y alcanzar sus objetivos con nuestros productos y servicios con contenido en el que pueden confiar. Explore nuestras recientes actualizaciones.

Hacer que el código abierto sea más inclusivo

Red Hat se compromete a reemplazar el lenguaje problemático en nuestro código, documentación y propiedades web. Para más detalles, consulte el Blog de Red Hat.

Acerca de Red Hat

Ofrecemos soluciones reforzadas que facilitan a las empresas trabajar en plataformas y entornos, desde el centro de datos central hasta el perímetro de la red.

Theme

© 2025 Red Hat