Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 2. Version 3.2.0 release notes

Red Hat AI Inference Server 3.2.0 release provides container images that optimizes inferencing with large language models (LLMs) for NVIDIA CUDA and AMD ROCm AI accelerators. The container images are available from registry.redhat.io:

  • registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.2.0
  • registry.redhat.io/rhaiis/vllm-rocm-rhel9:3.2.0

With Red Hat AI Inference Server, you can serve and inference models with higher performance, lower cost, and enterprise-grade stability and security. Red Hat AI Inference Server is built on the upstream, open source vLLM software project.

New versions of vLLM and LLM Compressor are included in this release:

The Red Hat AI Inference Server supported product and hardware configurations have been expanded. For more information, see Supported product and hardware configurations.

Expand
Table 2.1. AI accelerator performance highlights
FeatureBenefitSupported GPUs

Blackwell compute capability 12.0

Runs on NVIDIA RTX PRO 6000 Blackwell Server Edition supporting W8A8/FP8 kernels and related tuning

NVIDIA RTX PRO 6000 Blackwell Server Edition

ROCm improvements

Full‑graph capture for TritonAttention, quick All‑Reduce, and chunked pre‑fill

AMD ROCm

2.1. New models enabled
Copy link

Red Hat AI Inference Server 3.2.0 expands capabilities by enabling the following models added in vLLM v0.9.1:

  • LoRA support for InternVL
  • Magistral
  • Minicpm eagle support
  • NemotronH

The following models were added in vLLM v0.9.0:

  • dots1
  • Ernie 4.5
  • FalconH1
  • Gemma‑3
  • GLM‑4.1 V
  • GPT‑2 for Sequence Classification
  • Granite 4
  • Keye‑VL‑8B‑Preview
  • LlamaGuard4
  • MiMo-7B
  • MiniMax-M1
  • MiniMax-VL-01
  • Ovis 1.6, Ovis 2
  • Phi‑tiny‑MoE‑instruct
  • Qwen 3 Embedding & Reranker
  • Slim-MoE
  • Tarsier 2
  • Tencent HunYuan‑MoE‑V1

2.2. New developer features
Copy link

Improved scheduler performance
The vLLM scheduler API CachedRequestData class has been updated, resulting in improved performance for object and cached sampler‑ID stores.
CUDA graph execution
  • CUDA graph execution is now available for all FlashAttention-3 (FA3) and FlashMLA paths, including prefix‑caching.
  • New live CUDA graph capture progress bar makes debugging easier.
Scheduling
Priority scheduling is now implemented in the vLLM V1 engine.
Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat